AI for Humanitarian Action with Justin Spelhaug - Transcript

Talk 226 – MSFT AI for Humanitarian Action Interview Transcript

Sam Charrington: [00:00:00] Today’s episode kicks off a series of shows on the topic of AI for the benefit of society that we’re excited to have partnered with Microsoft to produce. In this show, we’re joined by Justin Spelhaug, General Manager of Technology for Social Impact at Microsoft. And our conversation, Justin and I discussed the company’s efforts in AI for Humanitarian Action, a program which extends grants to fund AI-powered projects focused on disaster response, the needs of children, protecting refugees and promoting respect for human rights. Justin and I cover Microsoft overall approach to technology for social impact, how his group helps mission-driven organizations best leverage technologies like AI, and how AI is being used at places like the World Bank, Operation Smile and Mission Measurement to create greater impact. Before we dive into the show, I’d like to thank Microsoft for their support and their sponsorship of this series. Microsoft is committed to ensuring the responsible development and use of AI and is empowering people around the world with intelligent technology to help solve previously intractable social challenges, spanning sustainability, accessibility and humanitarian action. Learn more about their plan at Microsoft.ai. Enjoy.

Sam Charrington: [00:01:51] All right, everyone. I am here with Justin Spelhaug. Justin is the General Manager of Technology for Social Impact at Microsoft. Justin, welcome to this week in machine learning and AI.

Justin Spelhaug: [00:02:01] It’s great to be here. Thanks for having me.

Sam Charrington: [00:02:03] Absolutely, absolutely. I am really looking forward to jumping in to this particular topic with you, discussing what Microsoft is up to as far as AI for Humanitarian Action. But before we do that, share a little bit about your background and how you got to working in this field.

Justin Spelhaug: [00:02:21] Well I have 21 years at a single company, that may be a record these days, at Microsoft. Wonderful company, and I’ve done many different things. I’ve worked in Microsoft research, I’ve worked in Asia in our field, I’ve worked in operations. But all throughout my career, I’ve been the chief social business advocate, agitato, depending on who you are and pushing, always, to find a way to create that nexus between technology, scalable business models, and social impact in the private sector context.

And so I’m very pleased to be doing what I’m doing today. Born and raised in Washington. A United States Marine, proudly served. University of Washington for the Huskies out there listening to this. And just real glad to be here.

Sam Charrington: [00:03:10] Fantastic. Technology for Social Impact, you’re a General Manager, that means that’s a business unit of sorts or a business unit period. What is that organization all about at Microsoft?

Justin Spelhaug: [00:03:22] So this business unit uniquely sits within Microsoft Philanthropies. And the focus is to help mission-driven organizations like nonprofits and the United Nations, The World Bank, use technology to advance their missions and to advance their impact.

What’s a little unique about our group in the industry are three key things. First is that we bring together philanthropies, technology and sales, commercial models, to support these mission organizations, nonprofits, on an end-to-end basis for the full spectrum of their needs.

Second, we’ve got a pretty deep innovation strategy, building new technologies in the area of AI, which we’ll talk about, but also in the area of dynamics, collaboration, Azure Core Services for nonprofits.

And third, we’re a social business, so all of the profit we generate, all the incremental profit this team generates is reinvested back into more impact, more services and support for nonprofits, more cash grants and donations to our philanthropies team.

Sam Charrington: [00:04:27] And so, artificial intelligence, how long has the organization been working with its constituent customer organizations on AI related work?

Justin Spelhaug: [00:04:38] As you know, at Microsoft, we’ve been working on AI since 1991 or longer. It’s always been in Bill’s mind as we set up Microsoft research. Our AI for Good Initiatives, so the social facing side of this, was launched about 18 months. And the first initiative there was AI for Earth, followed by AI for Accessibility.

And then most recently launched last September at the United Nations General Assembly was AI for Humanitarian Action. Combined, that’s a 115 million dollar investment that Microsoft is making across all three of those pillars over the next five years to try to drive impact.

Sam Charrington: [00:05:21] And so, why do you think AI is important for these types of organizations?

Justin Spelhaug: [00:05:26] I think it’s important, first, if you’ll indulge me to step back on what’s the broader context of the social issues that the world is facing today. We’ll detour there for one moment and then I’ll come back to that question more precisely.

Today, in the last 20 years, we’ve seen nearly a billion people move out of poverty. The international poverty line is defined as a $1.90 per day, and those billion people came largely in China, largely in India on the back of some really great economic success, and that’s pretty amazing.

But still, there’s 800 million people that live under $1.90 per day. There’s 124 million of those people that have severe food insecurity, life threatening food insecurity each and every day. And those issues combined with some really challenging geopolitical issues have created the biggest crisis in refugees and internally displaced people that we’ve seen since World War 2, with 68 million.

I know you’re close to those issues. 68 million people today without a place to call home, displaced. And when you step back and say, “What’s the world’s answer to these problems?” We look to the sustainable development goals. These are the things that were defined by the world and the United Nations to really paint the picture that we want to see by 2030.

Whether it’s equality or eliminating poverty, justice in rights, all of these issues are a part of the sustainable development goals. The problem is this, it costs between five and seven trillion dollars a year to fund these goals, and we’ve got about a 2.5 trillion dollar shortfall in funding these goals.

And so we’re not making progress in addressing these humanitarian disasters that are unfolding right in front of us today, the environmental disasters that are unfolding right in front of us today, and government is part of the solution, but they can’t stretch to cover that full 2.5 trillion dollars. Business has to step up.

And while we don’t have all of the answers at Microsoft, we’re trying to shift from a world of corporate social responsibility that maybe more narrowly defined to a world of total social impact. We’re building at the core of our business model, the ability to innovate, serve nonprofits, and create new innovations like AI for Humanitarian Action that get right at these issues and help organizations unlock challenges.

So let me provide one example and then we can dive more deeply into how AI is applied in this specific context that I’m talking about. Going back to that food crisis and hunger crisis, The World Bank is chartered with providing funding to communities that have food insecurity and famine issues. We’re working with The World Bank to use our AI models, AI engines and tuning their models, bringing together environmental data, socioeconomic data, population data, geopolitical data to help predict food insecurity at a community level and determine which communities are most at risk to reaching a food insecure state or even reaching full-blown famine before it happens.

If we can do that, if The World Bank can do that more precisely, they’re able to release funding ahead of a disaster and save some lives. And that’s how the power of technology, leveraging our resources within Azure Cognitive Services, our resources from Microsoft research and combining that with World Bank teams to solve this problem, can have a powerful and transformative impact on these kinds of issues.

Sam Charrington: [00:09:31] Do you have a sense for what an engagement like that looks like? Who are you engaging with at The World Bank? What types of information sources are you using to help create the AI that helps drive these predictions?

Justin Spelhaug: [00:09:47] The World Bank is getting ever more sophisticated in how they’re engaging the private sector in these kinds of ways and there are some roles in The World Bank precisely chartered to work with Microsoft and other private sector companies and figure out how we get the most out of their resources.

So that’s the entry point where they frame up a problem and they ask, “How can we help solve that problem? And are we willing to not only provide some free computing capacity, but more importantly, provide the data scientists necessary to help improve the quality and the predictive accuracy of the models?”

We put a proposal together, get that proposal on the table and then form a strategic partnership. That process, just to be clear, takes some time to really map out, what is the problem? What resources do they have? How can we augment those resources in a way that’s most effective?

And this particular case, The World Bank is asking for help with model tuning, affected in their ML models, to improve the predictive accuracy of the data they already have and the models that they already have, they just don’t have the accuracy where they want it to be yet.

And so that’s how we’re engaging, and so we’re bringing to bear resources that are experts in that area to supplement The World Bank team and help them get that over the goal line. One other thing that’s important to note about this particular project, and is unique in the industry, is that The World Bank is not only pulling from Microsoft, but they’re also asking Google to step up, AWS to step up, and it’s an interesting collaboration, in fact, between all three companies as we work to help them solve this problem.

These problems are bigger than any one brand or any one company and we have to learn how to collaborate effectively together with organizations like The World Bank to solve problems that, frankly, impact human lives.

Sam Charrington: [00:11:49] Maybe taking a step back from this particular application, is there a way that you categorize the various types of applications that you get involved in under the banner of humanitarian action?

Justin Spelhaug: [00:12:01] Yeah. In the year of humanitarian action, we’ve identified four core scenarios that we want to focus on. The first is disaster response. Essentially helping forecast disaster before it happens. That was the example I provided, but also helping first responders respond more precisely and effectively when disasters occur.

The second is needs of children and there’s a range of issues we’re focused on there from healthcare to protection and support. Third is protecting refugees and helping organizations that are serving refugees, like the Norwegian Refugee Council or the Danish Refugee Council or the UN scale their services more effectively.

And then the fourth is promoting respect for human rights and we’re very involved with topics like disrupting human trafficking, which your listeners may know is one of the largest criminal industries in the world today, and is a real challenge. All four of those areas we have examples we can talk about.

Sam Charrington: [00:13:09] Let’s do that. The refugee one is of particular interest to me since 2010 I’ve sat on a board of a local organization in St. Louis, the International Institute, which is one of 100 or so affiliates of the U.S. Committee for Refugees and Immigrants. And is one of the organizations that assists in the resettling of refugees into the St. Louis area.

And as a child of an immigrant, it’s always been a passion point for me. What are some of the applications of AI to addressing the issues that are faced by refugees around the world, really?

Justin Spelhaug: [00:13:51] I’ll give you two examples. One that’s kind of live and one that we’re working on. The Norwegian Refugee Council provides many things for refugees, but one of the things they provide is legal services.

If we pick a geography like Iraq where we’re dealing with refugees and displaced people, there are four million people in that country and the adjacent areas that need access to legal services.

Now, these legal services, for your listeners, are pretty basic things. It’s about getting identity for their newborn child, filing a death certificate, getting access to benefits that they’re not getting access to through legal channels.

But the Norwegian Refugee Council only has so many lawyers, and so the technology that we’re building with them is a chat bot based platform that allows for a more intelligent interaction with the refugees so that we can direct them more precisely to the specific legal services that they need within NRC, allowing their lawyers to scale more effectively.

Prior to the chat bot technology, the lawyers were having to triage each individual case and then route it to the correct specialist. With our technology, we’re able and working towards being able to do that in a much more precise way, reducing the lead time that it takes to render those legal services and allowing these lawyers on the NRC side to scale up.

With another organization, we’re working on actually child identification for reunification. And so, we talk a lot about AI and ethics and some of the challenges that we face around facial recognition, for sure. And those are real challenges. But in this case, we’re using facial recognition to really drive a positive outcome where we’re able, through machine vision and image matching, to measure the symmetry of a child’s face and match that to a database of potential parents.

That child may have been dislocated as there was a movement within the camp for one reason or another, they can’t find their parent in these camps and resettlements and they’re using technology to match-make that child back to that parent and drive reunification more efficiently.

Sam Charrington: [00:16:18] The legal example that you gave sounds very similar to the kinds of things we see on the commercial side with organizations using chat bots to provide support.

Justin Spelhaug: [00:16:27] That’s right.

Sam Charrington: [00:16:28] Are there particular challenges with this particular application of chat bot technology, of these kinds of technologies, that are unique to the specific use case?

Justin Spelhaug: [00:16:43] I don’t know so much as challenges as there are a lot of demand. The one thing a nonprofit is constrained on is resource. They’re typically not constrained on the demand for their service given the magnitude of the challenges they’re working.

We’re working with another organization, as an example, on an emergency services chat bot, so call it 2-1-1 chat bot. That allows this organization to better direct their beneficiaries to the right emergency services in time of disaster, and prior to the implementation of this technology … And this one is actually in flight right now so it’s not fully implemented. Their 2-1-1 lines were just getting overwhelmed.

And so I think we’re seeing many different applications of chat bot technology that allows nonprofits to scale their services and more precisely, it’s all about matching, more precisely matching the demand in the moment to the expert that can solve it without that intermediary triage process that nonprofits can’t afford to do. And so there’s a real efficiency gain there.

Sam Charrington: [00:17:53] Does the typical nonprofit have the technical sophistication to be able to absorb the solutions that you’re proposing for them? Or does that create an ongoing challenge for them, because these AI systems, they need maintenance-

Justin Spelhaug: [00:18:12] They do.

Sam Charrington: [00:18:13] … you can’t just throw them out there and they’ll run forever without fine tuning and ongoing maintenance. What kinds of challenges do you see there in delivering this kind of technology to nonprofits?

Justin Spelhaug: [00:18:29] Great question. Nonprofit is a tax code and so, you have this absolutely enormous spectrum of organizations. There are about four million nonprofits, we believe. 99 percent of those organizations are less than 50 people. Half of those organizations have anybody in IT or any formal IT funding.

And so, there is very limited capacity in the ecosystem overall, kind of number one. So then how do we think about AI and Humanitarian Action? Well, one of the qualifying criteria for applying for a grant and leveraging the services that we’re providing is that you do have resources on the other end, that not only can maintain the model, but can work with us to build and organize the model.

You have the data and the data is also available because it can’t do much without the data. And so an organization needs to be ready to use these kinds of tools. We do envision a world though … I should mention every one of these tools, like the 2-1-1 chat bot capability or the NRC chat bot capability that I talked about, we’re abstracting and putting those patterns and practices and code on GitHub.

So any other organization, nonprofit organization, that may be of the same or similar scenario can access that, can download it, and can start using it. We’re making investments with nonprofits to build their technology capacity. We imagine a world where in the future, nonprofits are able to go to GitHub, they’re able to get training, and they’re able to start using these technologies, but we’ve got a long way to go. There’s only a fraction of the organizations today that have the capacity to really put this into practice.

Sam Charrington: [00:20:19] The talent aspect of this is a huge challenge. It’s often difficult for large enterprises to compete with the Microsoft’s of the world, the Googles of the world for talent or the Silicon Valley companies for talent. And it’s even harder for the typical nonprofit.

Justin Spelhaug: [00:20:39] Absolutely.

Sam Charrington: [00:20:40] And so, I’m curious, you mentioned education, is that a big part of your charter, to provide educational resources to these types of organizations?

Justin Spelhaug: [00:20:52] It is. In fact, for anybody in the nonprofit community listening, I think it’d be fair to say most people work in a nonprofit environment because they are mission driven and they believe in the mission.

And that’s why you get paid less often and still work 60, 70, 80 hours a week, because you’re passionate about refugees or the environment or health or child protection.

And part of my charter is building capacity in these nonprofit organizations. And actually building capacity with their beneficiaries. We’re working on a number of different programs using Imagine Academy and other content we have and creating some new offerings that will help do that for nonprofits in a very affordable way.

And we’ve made investments with organizations like UNICEF, as an example that, as you know, has a mission around child protection to create platforms that will deliver digital skills as well as a broad range of educational experiences for 75 million children around the world who are on the move. Children that are migrants, internally displaced refugees.

So we’re focused actually at both ends, to build capacity at both ends through our philanthropic tool … philanthropic programs, rather.

Sam Charrington: [00:22:09] So we talked through some examples on the refugee side. On the disaster recovery side, what have you seen there?

Justin Spelhaug: [00:22:18] I highlighted two. One was that World Bank famine prediction tool. Another is helping organizations respond to demand. That’s the 2-1-1 platform that we talked about. But we’re also helping organizations organize their volunteers more effectively.

So there’s one organization here in the United States that mobilizes a whole lot of volunteers during disaster time, particularly when earthquakes strike or hurricanes hit. And you need to know what your volunteers are certified on. What equipment can they use? Can they use chainsaws? Can they use forklifts? Can they drive bulldozers to help clear the debris, to help rebuild?

We’re using OCR technology to automate the assessment and identification of certifications for their volunteer base that extends well beyond 80,000 volunteers all around the nation. So they know precisely who has which skill and can be deployed to which location.

We’re also working with an organization that’s working on an open mapping platform. Now this open mapping platform is used during times of disasters to help first responders pinpoint where to focus. But as you know, the map before a disaster is very different than a map after a disaster.

And we’re using AI and some ML models to better ingest images and assess buildings and building damage to figure out what’s changed pre-disaster to post-disaster most significantly, relative to building structure. And how should first responders plan their engagements and their interventions based on where we see the most damage. So that’s another example of technology that we’re building there.

Sam Charrington: [00:24:15] Can you elaborate on that one? Is that based on vision or-

Justin Spelhaug: [00:24:18] It is, yeah.

Sam Charrington: [00:24:19] … satellite imagery or what are the data sources there?

Justin Spelhaug: [00:24:23] Let me explain the process here. This organization uses volunteers, and I’m saying “this organization” because we’re not public yet, so that’s why whenever I say that, it’s because it’s not a public case yet.

Uses volunteers to take new satellite imagery and trace that onto the map. They have local volunteers that add additional details to the map. Neighborhoods, street names, buildings, evacuation centers, and then humanitarian organizations use that in response.

Now we’re adding machine vision into that equation to automate and expedite the damage assessment on buildings, as an example, in that process. And so that first responders have this continually updating feed of mapping information that’s … Some of it’s being generated by AI using machine vision to access buildings, some of it’s being generated by local volunteers that are drawing a street or a street no longer exists and says, “This is where the school was.” And that allows these first responders to pinpoint damage much more effectively and in real time to deploy those resources in the right way.

Sam Charrington: [00:25:32] It makes me think of just the use of maps. We all in navigating our daily lives now use maps constantly and in a situation where a disaster has occurred, those maps aren’t really useful. So in addition to assessing building damage, just the ability to effectively route in an environment like that has to be a challenge.

I don’t know if that’s part of the initiative that you’re working on with this organization-

Justin Spelhaug: [00:25:59] Well-

Sam Charrington: [00:25:59] … but-

Justin Spelhaug: [00:26:03] Resource optimization as a topic is … That is where AI in many ways … 20 years ago, my first AI project was supply chain optimization using a platform, and that is a core and critical issue for organization.

To give you an example of how we’re applying that technology, we’re actually applying that technology in supporting refugees. And this is a public case, this is with the Danish Refugee Council who’s deploying Dynamics 365 finance and operations.

So they’ve deployed that module and we’ve deployed that module so that we’re able to optimize how they’re delivering aid, food, water, wash, basic shelter. Because if they can get that aid delivered on time and accurately around the camps that they’re managing, and believe me it’s a tough problem because demands are always changing, supply positions are always changing, and if you don’t get it right, lives are at stake.

They’ve got to get that right, and when they do, they can focus on providing higher level services, counseling, job creation, business creation, once they get those basic needs taken care of. So building AI, as you know, AI’s not just a thing that’s Azure Cognitive Services, it’s woven into the fabric of all of our platforms at Microsoft, including Dynamics.

And back to your question of, “Well, how can nonprofits really take advantage of AI?” I’m highlighting some really specialized use cases here, but if you think about AI built into our Microsoft 365 platforms, or Dynamics platforms in this particular example, it becomes a much more palatable engine to leverage for nonprofits in the way that they’re running their missions and optimizing their distribution of supply, in this case.

Sam Charrington: [00:27:55] So you’ve already mentioned the Danish organization working with refugees, a Norwegian organization working with refugees and I think in both cases, they were managing camps. Do you happen to have any stats on the number of camps, the number of people in camps? Just the scope of the challenge on a global basis?

Justin Spelhaug: [00:28:16] I don’t have the number of camps. I was just in a camp in Kakuma, a northeast corner of Kenya. We flew in to really get on the ground and understand the issues of the camp. You get on the ground and there’s 20 … In Kakuma, there are between 28 and 32 different NGOs all surveying this camp.

Now, Kakuma was 150,000 people in this camp, and it was built for a capacity of about 100,000. So it was bursting at the seams. And the opportunity that we saw there was how do we start to help these organizations think about a common data model and a common set of platforms, Microsoft or non Microsoft, that can interoperate so that when their serving refugees, they’re able to coordinate better?

So Amid, who may be a refugee there in Kakuma, we’re able to understand all of the services and all of the organizations that are supporting him and which interventions are helping him get access to economic opportunity and a better livelihood.

Back to your question though, I mentioned that there are 68 million displaced people and refugees. We can fact check this, but it’s about 28 million refugees and 40 million internally displaced people. Most of them are not in camps. Most of them are in urban environments and in cities.

And that makes the challenge that much harder for these refugee aid organizations like the Danish Refugee Council or the Norwegian Refugee Council to support them, but the services that we’re building, these chat bot services, the predictive services that we’re building will operate within the confines of a camp or operate also in an urban environment.

Sam Charrington: [00:30:00] So we’ve talked about two of the four already. Why don’t we talk a little bit about the needs of children work you’re doing?

Justin Spelhaug: [00:30:10] Perfect. Yeah, this is a really interesting category of work that we’re doing, and it does intersect with the work we’re doing on human rights as well. So you’ll see that bleed over.

One area of work we’re doing is with an organization called Operation Smile. Operation Smile works with children in low income markets and communities around the world and provides cleft palate and cleft lip surgery. For the listeners that may not know what that is, that is a facial deformity that children are born with that impacts their ability, in fact, to latch onto their mother and get nutrients.

It impacts medical conditions like hearing. Creates, obviously, dental issues, but it also creates a massive social stigma for these children. The challenge that Operation Smile has is their mobilizing plastic surgeons from all over the world, from Johannesburg to LA, to provide surgeries, life-changing surgeries for these children.

But these surgeons are operating in hospitals that they’re not accustomed to. They’re operating with equipment that they may not be using every day, and they’re doing surgeries that are unique for them, often. And so they need feedback and they need feedback in real time.

The old process that Operation Smile had was to take a picture of the child before surgery, take a picture after surgery, send it to an evaluator, and a month later the doctor got feedback. The new process that we’ve implemented is we’re able to take a picture before the surgery, take a picture after the surgery, and right in that moment, we’re using machine vision to actually score the severity prior to surgery of that cleft palate or cleft lip, how severe is it?

We score the quality of the post-surgical photo and we’re analyzing a whole bunch of dimensions, looking at facial symmetry, how clean the cleft palate is now connected, and there’s a score that’s provided post-surgery. That’s brought into a database in a Power BI view where we’re able to see the scores of all other surgeons performing a similar procedure.

Does two things, provides that surgeon feedback in real time on how they’re doing, but also allows them to match make a mentor that may be doing a better job on that particular surgical procedure, get on the phone before their next surgery, which may be scheduled in 45 minutes, get some tips on the technique, and then go back into that theater and improve that next child’s outcome right then and right there. So that’s a pretty cool case.

Sam Charrington: [00:33:01] It is, and taking a step back, we’ve kind of already honed in on a couple of, I don’t know, use case areas for AI. We talked about this resource optimization in the sense of mapping or making these organizations more internally efficient. And this use case as well as the volunteer onboarding one kind of speak to the ability to use AI to better connect people within an organization or better allow these organizations to better make use of their human resources.

I guess in this case, the volunteers may or may … the surgeons may or may not be working on a volunteer basis.

Justin Spelhaug: [00:33:48] They’re all volunteers.

Sam Charrington: [00:33:49] They’re all volunteers?

Justin Spelhaug: [00:33:49] Yeah.

Sam Charrington: [00:33:50] So it’s another volunteer management type of an application, and it also has echoes onto the enterprise.

Justin Spelhaug: [00:33:59] It does.

Sam Charrington: [00:34:01] I’m really just making an observation on what I’m hearing.

Justin Spelhaug: [00:34:04] No, you’re helping me with our strategy, in fact. I love it. I love it. And maybe there’s a third use scenario here, which is the next one in terms of the needs of children. And this is about deep learning. This is about using Bayesian networks and deep learning to try to identify patterns that we couldn’t previously see.

Infant mortality is still a major, major, major issue in both developed markets and developing markets, more in developing. And SIDS, sudden infant death syndrome, is one of the number one drivers of infant mortality. It’s been around forever and it’s incredible how little we know about what’s driving SIDS. With a partnership-

Sam Charrington: [00:34:51] What’s driving its prevalence or what’s the underlying cause?

Justin Spelhaug: [00:34:54] What’s the underlying cause. Well, both, actually. What are the risk factors that create a propensity for SIDS? And then, when we identify those risk factors, what can we do differently with that child in order to minimize the potential outcomes?

So in partnership with the Seattle Children’s Hospital, our data science team, led by John Kahan, who’s very passionate about this topic. In fact, he lost a son to SIDS. Pulled a massive database that existed and it turns out that we have a record for every child born in the United States that dates back to … I’ll get the date wrong, but it’s somewhere in the 80s.

And that record has a whole bunch of attributes in terms of child’s weight, height, has a wide range of attributes, and those attributes were never really fully analyzed to figure out what was the causation and correlation between all these different attributes and sudden infant death outcomes.

The team ingested all of this data into Azure and used our machine learning tools and in particular, one particular Bayesian network technique to analyze the data and to start to understand the correlative factors.

Through this analysis, they’re able to get a whole bunch of insights and I think it’ll be worth bringing John on the show to talk about the depth of the insights, but able to pinpoint down to the cigarette how it increases the risk factor for a child, an individual cigarette, increases the risk factor for a child relative to SIDS.

This body of work has also led to new innovations that they’re pursuing. We know that a child with SIDS is at much higher risk when it lays on its tummy. But when you’re a parent, you’re tired, you haven’t slept for days, you’re out of the room, how do you know if your child’s on their stomach?

Well, John’s looking into machine vision technology that’s able to recognize whether a child’s on their back or their stomach and provide an alert to the parents so they can flip the child back over. And so, a couple of really neat use cases and ongoing analysis. This analysis has just started and the fruit has just started to be born from the work. That will hopefully help really provide effective remedies to SIDS long range. So those are two examples of the work that we’re doing in needs of children and maybe another scenario here that we’re talking about.

Sam Charrington: [00:37:43] I’m curious about that last one. I’ve talked to other folks working in clinical environments or with data captured at hospitals and it’s historically very difficult to get access to for a variety of concerns, most obvious being privacy concerns.

I’m wondering if you have any additional insights or context as to what the process was like in the context of this project?

Justin Spelhaug: [00:38:12] I think John would be probably the best to answer that, but the research was conducted with a partner, Seattle Children’s Hospital. I think a lot of it was executed through the partner and we provided the AI capabilities on the backend, but John would be able to provide-

Sam Charrington: [00:38:25] Got it.

Justin Spelhaug: [00:38:25] … every detail on that topic.

Sam Charrington: [00:38:30] I’m sure. Okay, so human rights.

Justin Spelhaug: [00:38:33] Yeah. So human rights, this is a wide topic. To give you a sense of what we’re doing there, I’ll give you three quick examples. Social sentiment analysis is something that we use in marketing each and every day. Understanding feeds from the news, understanding feeds from social networks and understanding what they’re telling us about our company. But what are they telling us about our cause?

If our cause is to understand capital punishment and where capital punishment is occurring around the world and whether there’s been due process around capital punishment, sometimes you need to fine tune those social sensing tools.

So we’re working with one organization, that’s their core business model, is advocacy for human rights and giving them a platform to analyze all of these data feeds and to leverage a keyword model, like capital punishment, death, execution and many other words that we use, to start to understand what is the frequency that we’re seeing around the world. In countries that have historically under reported it, well, guess what? It’s typically in the news or it’s in the social blogosphere.

And what is the sentiment in that country and how can they better shape their advocacy? So that’s one example of what we’re doing in human rights.

Another example is we’re working with an organization to analyze Syrian war crimes videos and there we’re using machine vision to identify and match known war criminals to crimes that are captured through this footage.

And then the third, and maybe more intricate example here is the work we’re doing on human trafficking. And human trafficking is a massive, massive criminal industry and problem. And particularly, sex trafficking is. It’s something that doesn’t seem to get talked about enough given how pervasive the challenge is.

Here we’re using technology to both disrupt the demand for sex work as well as the supply. And the way we’re doing that, in partnership with organizations like Seattle Against Slavery, and other organizations, we’re using chat bot technology on the demand side of this challenge to engage would-be buyers. And our chat bot poses as a sex worker, engages that would-be buyer in a dialogue, and ultimately based on that conversation, will direct that person to resources, but also let that person know that they’ve been engaged by an anti sex trafficking organization.

So that’s on the demand side. On the supply side, typically these girls and boys who are in this industry are posting their services in a wide range of places on the web and on the dark web. Well, we’re building tools that allow organizations like Seattle Against Slavery to scrape and find these individuals.

Typically these individuals will have multiple phone numbers and they’ll encode those phone numbers with different hashtags and symbols so that they’re not easily machine readable. Well, we’re able to crack that and decode that, and that provides social workers the … We created a platform that provides social workers the ability to blast messages out to this community of sex workers to let them know there’s resources. To let them know that there’s somebody on the other end of the line who will help provide them support.

And from those engagements, which are typically done over text messaging, set up an intervention meeting where they can help these folks get the services they need to get out of the sex trafficking industry, get the protective services they need, and change the course of their life. So an interesting blend of technologies there used by organizations.

Sam Charrington: [00:43:07] Yeah, based on my experience with working with nonprofits, one of the things they are very good at is assessment. Assessment of their own programs and projects and their ability to actually make impact on the communities that they serve. Assessment and evaluation is also critical for AI based systems.

I’m curious whether in your experience there are unique challenges associated with the ongoing assessment of the effectiveness of AI in these contexts? Or are the evaluation challenges the same as you might find in enterprise context or other places, or are there … Do you need to work with these organizations to connect the way they measure the success of their programs with the AI tooling?

Justin Spelhaug: [00:44:00] Yeah. Just to back up for one second, I think many organizations still struggle with connecting their activity to the end outcome. For those refugees we’ve been talking about during this conversation, we want to improve their livelihood, we want them to have jobs, we want them to have a productive life and a productive family.

But oftentimes, organizations are stuck measuring how many aid packages did we deliver to this location? Not, did we transform somebody’s life? We just kind of make that assumptive step. I think there’s two things that we’re actually doing in that area to close that gap.

One is, in our Dynamics 365 platform, we’ve been making some really significant investments in that platform for nonprofits. And it’s important, when you think about Dynamics 365, to think about just how critical it is for nonprofits. At the front end of a nonprofit, you’re fundraising, you’re managing your volunteers. Those are your core resources. You’re managing beneficiaries and beneficiary cases.

You’re delivering programs and measuring their impact, all of that is based and rooted in CRM logic and capabilities. On the backend, you’re managing finance, operations, HR, all of that is rooted in ERP logic and capability.

And what you ultimately want to understand is, what is the cost or the efficacy per outcome that I’m trying to generate? And that is a really hard bridge to cross. So to cross that, we’ve invested in a common data model on top of Dynamics 365 for the nonprofit industry, designing it specifically for the industry, with the industry. So we have institutional donors, private donors, like Gates, [[DFID (Department for International Development?]][inaudible 00:44:05], large nonprofits and small nonprofits that have helped us co-create this common data model.

The common data model sits on GitHub so that the world can use it. And so that we’re not the only platform running it, ’cause the more platforms that consume that data model, the better and more interoperable we can make those platforms.

Sam Charrington: [00:46:15] I mean, can you use it anywhere, not just Dynamics?

Justin Spelhaug: [00:46:18] Yeah, it’s on GitHub. It’s a schema. But Dynamics consumes it as a first class citizen, of course. We built it, we want to consume it and use it. On top of that data model, we’re building connectors, templates, and sample laps that are really igniting partners to build really cool finish solutions.

But one thing that’s core in the data model is unlike other models that have been built before, they’ve been built really with fundraising in mind. They started as a CRM sales management tool and built with fundraising in mind. We built ours with program management, program delivery, and outcome measurement in mind. So very specifically, and we’re doing a lot of work on benchmarking best practices on outcome management, outcome metrics, as well as program, that’s built-in to this data model.

On top of that, as I was saying, partners like Avanade, like Blackbaud, Classy, Fluxx and others, m-hance[a][b][c], are building solutions, finished solutions that they can deliver to nonprofits. We think there are some very interesting AI scenarios, to get back to the topic, to leverage the data in those models and leverage our intelligent cloud computer infrastructure to help nonprofits be able to draw a straighter line between that package of aid that they delivered to that refugee camp, to refugee outcomes. And the cost per outcome.

If you’d say, “What’s the one thing that you’re working on, what’s the single most important thing that you’re trying to do in your team right now?” There’s many, but it is helping organizations much more clearly understand that causation and correlation between their activities, the outcomes they’re driving, so they can tune and modify their programs to have an even greater outcome. And AI is key to that.

Sam Charrington: [00:48:11] I agree, and you did a great job at picking up on the kernel of an idea that I was trying to get out there. You tend to see a lot of activity focused on evaluation, mostly because it’s part of … Many of these organizations are grant funded, the grantors all have program evaluation components of their grant, so they are kind of constantly working on evaluation, but-

Justin Spelhaug: [00:48:39] On a treadmill, in fact.

Sam Charrington: [00:48:40] On a treadmill, right. But there is this gap between the metrics that they have access to and can use and the actual impact to the point that you’re, to your point, that they hope to make in the whatever community that they’re serving. And it does strike me that AI could play a huge role there in its ability to look in a broad spectrum way at this constituent and identify patterns, right?

Justin Spelhaug: [00:49:14] That’s right.

Sam Charrington: [00:49:14] A big part of what deep learning, for example, is great at is … or situations where we can’t really figure out the rules-

Justin Spelhaug: [00:49:20] It’s fuzzy.

Sam Charrington: [00:49:21] It’s fuzzy. But we can, based on data, we can train a model to identify the success case. That would be huge for many organizations.

Justin Spelhaug: [00:49:32] It is, and just two things to add to that. This morning I was on a call with the CEO of Mission Measurement, and what Mission Measurement is focused on is what he calls the outcome genome, but essentially a framework of outcomes that really denote social impact.

And if you go to Mission Measurement’s website, you’ll see on the website they have 132 outcomes predict and correlate to 80 percent of the social impact that we drive. So it’s a standard dictionary for how we can think about these outcomes by industry.

You map that to a common data model and a deep data repository with these nonprofits, and in between, you’re able to leverage AI. Now you’re cooking with gas, in terms of helping organizations really understand the impact you’re having, and you have a common language to start to describe globally what is the dollar per outcome that organization A is able to provide and organization B is able to provide, and how do we help them optimize those things?

We’re actually applying that same genome to our work. So the question I ask myself is, “Hey, how is Dynamics? How are these AI solutions actually moving the dial? Great stories, Justin, but how many kids were actually … How many life changing surgeries actually improved because of those optical recognition technologies that you’re talking about?” We’re working on building out that platform capability to tell our story too.

Sam Charrington: [00:51:06] Okay. A couple of questions for you, or maybe one question from a couple of perspectives. And the question is, to your point, lots of great stories here, lots of great opportunity to apply AI for humanitarian action and for social good more broadly. I’m wondering if I’m listening to this podcast and I’m at a nonprofit or an organization that’s focused on providing services to these communities, or other communities, how do I get started from your perspective, and taking advantage of AI?

And then the flip side of that is if I’m an individual that’s not working on your team at Microsoft or not working in one of these organizations, but sees the need, maybe has some skill in this area and wants to jump in, I’m curious what … If you’ve seen anything, any organizations that you work with or any suggestions you would have for folks that want to help?

Justin Spelhaug: [00:52:12] Just to peel back that question, the starting point for AI, I think, is having a well-defined use case. And we kind of get fixated on the shiny object of AI and all of its glory, but we can often forget that what we’re really trying to do is solve problems here.

And so, as an organization thinks about getting started, what are the problems they’re trying to solve? What analysis are they trying to predict or understand? And The World Bank is a really good example. There was a pretty well defined use case, it was about food insecurity prediction with core dataset and a common model.

The second thing an organization needs to have as they start to head down the AI path is data and data and data. Now there are ways to solve AI problems with some advance modeling techniques, but in general, we need to solve these challenges, a pretty significant dataset. A pretty decent dataset.

And so, do you have a well-defined use case, and do you have the dataset behind that use case? And then the third is starting to build some skill. We’re putting as a company, a number of different programs together to build skills and training for folks, but I do think we have an opportunity to work with partners in the sector, Revel, KPMG, Avanade are all building competencies.

They all have social business practices. Accenture has a social business practice to help nonprofits supplement their skills with great AI talent. And then everything that we’re building here, everything that we can, we’re gonna be putting on GitHub.

The end game, in my view, is that we’ve got a toolkit of hundreds of tools. Not all that we have built, but the community is building too, that can be compiled together into this toolkit in AI for Humanitarian Action, four core scenarios, depending on what you’re working on, you can pull from that toolkit to either be inspired or leverage that AI pattern directly for your particular use case.

There are no magic bullets though. I mean, there’s no magic bullet other than, I think, those four or five things.

Sam Charrington: [00:54:51] It does strike me in thinking about the organizations and their constituents that we’re working with potentially under resourced organizations or less sophisticated organizations. Not always, but often. And vulnerable communities. That brings to mind the whole concept of ethical use of AI and I would imagine you would …

As an organization that’s bringing this technology to them, you have a responsibility to ensure that they are using it correctly. To what extent does that come up and how have you addressed that in your engagements?

Justin Spelhaug: [00:55:39] Yeah. No, I mean, ethics in AI is a very hot issue at Microsoft in general, as you know. Part of our work is to demonstrate to the world the good that AI can do, which is why we have initiatives like AI for Humanitarian Action.

But at Microsoft, we’ve invested in AI for Ethics framework and that framework was published out by Brad Smith and Harry Shum and The Future Computed body of work. It’s a book, you can find it online. It outlines six core principles. I won’t go into them in detail, but fairness, reliability, privacy and security, inclusiveness, transparency, and accountability are all, in each of those, is kind of a discussion, I think.

They’re all core to designing AI in a way that respects the security and the privacy of that SIDS patient record that we talked about earlier or that refugee record that we talked about earlier. Now, that framework is being put together and has been put together into an ethical design guide that all of our engineering teams adhere to as we think about building products or services or engaging in these projects, as well.

We also have an internal advisory committee that looks at major new product releases, new technology releases, and ensures that they follow those ethical standards, that we’re designing it in the right way. In short, we have to apply those to the products that we’re building, but we also have to apply those to the engagements that we’re leading, and we do. And that serves as our north star compass point to make sure that we’re doing things that are advancing society, that are inclusive for everyone, that are safe and secure.

Sam Charrington: [00:57:27] Well, Justin, I’m really excited about the work you’re doing, and I appreciate you taking the time to chat with us about it. Any final thoughts you’d like to share?

Justin Spelhaug: [00:57:36] Well just that we’re really excited to support the nonprofit community, these mission-driven organizations. We’re gonna leverage that social business model that we have to continue to dial our investments up and into this community, building more AI patterns and practices, investing in Dynamics to make it increasingly more useful and potent. Helping benchmark best practices and processes for the sector and contribute that to the sector.

And we’re really excited about what that can do. And ultimately, we’re excited about working with these organizations to move that dial on that first thing that we talked about, which are those sustainable development goals, and really leveraging everything that we can do at Microsoft, our products, our technology, our people, to lean in with these mission-driven organizations to close that gap we see. And if we do our job right, we’ll be sitting here in 2030 talking about a little bit of a different, and hopefully a much better, world.

Sam Charrington: [00:58:42] Fantastic. Thanks so much, Justin.

Justin Spelhaug: [00:58:43] All right. Thank you.