We could not locate the page you were looking for.

Below we have generated a list of search results based on the page you were trying to reach.

404 Error
Akshita is a Senior Research Engineer on the AllenNLP team, involved in R&D for natural language processing (NLP). Most recently, Akshita has been working on the OLMo project, where she has contributed to pretraining dataset construction, model training and inference, and evaluation tools and benchmark. She has also worked on open-source libraries such as allennlp, ai2-tango, etc. Akshita graduated with a Master’s degree in Computer Science from the University of Massachusetts Amherst in 2020, where she worked with Prof. Mohit Iyyer at the intersection of NLP and digital humanities. Previously, Akshita worked at Cerebellum Capital (Summer 2019), and at InFoCusp (2015-2018), where she worked on building a data science platform. In her spare time, Akshita enjoys reading novels, writing (especially poetry), and dancing.
I am a Research Scientist at Google. Previously, I completed my Ph.D. at Boston University, advised by Professor and Dean of the College of Arts and Sciences Stan Sclaroff. My primary research focus is computer vision and machine learning. I interned at Amazon working with Javier Romero, Timo Bolkart, Ming C. Lin, and Raja Bala during the Summer of 2021. I interned at Apple AI Research during the 2019 and 2020 Summers where I worked with Dr. Barry-John Theobald and Dr. Nicholas Apostoloff. In 2018 I was a Spring/Summer intern at the NEC-Labs Media Analytics Department, where I worked with Prof. Manmohan Chandraker and Dr. Samuel Schulter. I graduated from Georgia Tech in Fall 2017 with an M.Sc. in Computer Science specializing in Machine Learning, advised by Prof. James Rehg at the Center for Behavioral Imaging. Recently, our work DreamBooth has been selected for a Student Best Paper Honorable Mention Award at CVPR 2023 (0.25% award rate) I have been selected as a Twitch Research Fellowship finalist for the year 2020 and as a second-round interviewee for the Open Phil AI Fellowship. I also appeared on the popular Machine Learning and AI podcast TWIML AI talking about my recent work on defending against deepfakes. While on a 5-year valedictorian scholarship, I obtained my B.Sc. and M.Sc. from Ecole Polytechnique in Paris, France. Additionally, I worked as an intern at MIT CSAIL with Dr. Kalyan Veeramachaneni and Dr. Lalana Kagal.
Naila Murray obtained a BSE in electrical engineering from Princeton University in 2007. In 2012, she received her Ph.D. from the Universitat Autonoma de Barcelona, in affiliation with the Computer Vision Center. She joined Xerox Research Centre Europe in 2013 as a research scientist in the computer vision team, working on topics including fine-grained visual categorization, image retrieval, and visual attention. From 2015 to 2019, she led the computer vision team at Xerox Research Centre Europe and continued to serve in this role after its acquisition and transition to becoming NAVER LABS Europe. In 2019, she became the director of science at NAVER LABS Europe. In 2020, she joined Meta AI’s FAIR team, where she served as a senior research manager. She now leads as the director of AI research at Meta. She has served as area chair for ICLR 2018, ICCV 2019, ICLR 2019, CVPR 2020, ECCV 2020, and CVPR 2022, and program chair for ICLR 2021. Her current research interests include few-shot learning and domain adaptation.
Recognized worldwide as one of the leading experts in artificial intelligence, Yoshua Bengio is most known for his pioneering work in deep learning, earning him the 2018 A.M. Turing Award, “the Nobel Prize of Computing,” with Geoffrey Hinton and Yann LeCun. He is a full professor at Université de Montréal, and the founder and scientific director of Mila – Quebec AI Institute. He co-directs the CIFAR Learning in Machines & Brains program as a senior fellow and acts as scientific director of IVADO. In 2019, he was awarded the prestigious Killam Prize and in 2022, became the most-cited computer scientist in the world. He is a fellow of both the Royal Society of London and Canada, Knight of the Legion of Honor of France, Officer of the Order of Canada, Member of the UN’s Scientific Advisory Board for Independent Advice on Breakthroughs in Science and Technology since 2023 and a Canada CIFAR AI Chair. Concerned about the social impact of AI, he actively contributed to the Montreal Declaration for the Responsible Development of Artificial Intelligence.
Dr. Sameer Singh is an Associate Professor of Computer Science at the University of California, Irvine (UCI). He is working primarily on robustness and interpretability of machine learning algorithms, along with models that reason with text and structure for natural language processing. Sameer was a postdoctoral researcher at the University of Washington and received his PhD from the University of Massachusetts, Amherst, during which he interned at Microsoft Research, Google Research, and Yahoo! Labs. He has received the NSF CAREER award, selected as a DARPA Riser, UCI ICS Mid-Career Excellence in research award, and the Hellman and the Noyce Faculty Fellowships. His group has received funding from Allen Institute for AI, Amazon, NSF, DARPA, Adobe Research, Hasso Plattner Institute, NEC, Base 11, and FICO. Sameer has published extensively at machine learning and natural language processing venues, including paper awards at KDD 2016, ACL 2018, EMNLP 2019, AKBC 2020, and ACL 2020.
There are few things I love more than cuddling up with an exciting new book. There are always more things I want to learn than time I have in the day, and I think books are such a fun, long-form way of engaging (one where I won’t be tempted to check Twitter partway through). This book roundup is a selection from the last few years of TWIML guests, counting only the ones related to ML/AI published in the past 10 years. We hope that some of their insights are useful to you! If you liked their book or want to hear more about them before taking the leap into longform writing, check out the accompanying podcast episode (linked on the guest’s name). (Note: These links are affiliate links, which means that ordering through them helps support our show!) Adversarial ML Generative Adversarial Learning: Architectures and Applications (2022), Jürgen Schmidhuber AI Ethics Sex, Race, and Robots: How to Be Human in the Age of AI (2019), Ayanna Howard Ethics and Data Science (2018), Hilary Mason AI Sci-Fi AI 2041: Ten Visions for Our Future (2021), Kai-Fu Lee AI Analysis AI Superpowers: China, Silicon Valley, And The New World Order (2018), Kai-Fu Lee Rebooting AI: Building Artificial Intelligence We Can Trust (2019), Gary Marcus Artificial Unintelligence: How Computers Misunderstand the World (The MIT Press) (2019), Meredith Broussard Complexity: A Guided Tour (2011), Melanie Mitchell Artificial Intelligence: A Guide for Thinking Humans (2019), Melanie Mitchell Career Insights My Journey into AI (2018), Kai-Fu Lee Build a Career in Data Science (2020), Jacqueline Nolis Computational Neuroscience The Computational Brain (2016), Terrence Sejnowski Computer Vision Large-Scale Visual Geo-Localization (Advances in Computer Vision and Pattern Recognition) (2016), Amir Zamir Image Understanding using Sparse Representations (2014), Pavan Turaga Visual Attributes (Advances in Computer Vision and Pattern Recognition) (2017), Devi Parikh Crowdsourcing in Computer Vision (Foundations and Trends(r) in Computer Graphics and Vision) (2016), Adriana Kovashka Riemannian Computing in Computer Vision (2015), Pavan Turaga Databases Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases (2021), Xin Luna Dong Big Data Integration (Synthesis Lectures on Data Management) (2015), Xin Luna Dong Deep Learning The Deep Learning Revolution (2016), Terrence Sejnowski Dive into Deep Learning (2021), Zachary Lipton Introduction to Machine Learning A Course in Machine Learning (2020), Hal Daume III Approaching (Almost) Any Machine Learning Problem (2020), Abhishek Thakur Building Machine Learning Powered Applications: Going from Idea to Product (2020), Emmanuel Ameisen ML Organization Data Driven (2015), Hilary Mason The AI Organization: Learn from Real Companies and Microsoft’s Journey How to Redefine Your Organization with AI (2019), David Carmona MLOps Effective Data Science Infrastructure: How to make data scientists productive (2022), Ville Tuulos Model Specifics An Introduction to Variational Autoencoders (Foundations and Trends(r) in Machine Learning) (2019), Max Welling NLP Linguistic Fundamentals for Natural Language Processing II: 100 Essentials from Semantics and Pragmatics (2013), Emily M. Bender Robotics What to Expect When You’re Expecting Robots (2021), Julie Shah The New Breed: What Our History with Animals Reveals about Our Future with Robots (2021), Kate Darling Software How To Kernel-based Approximation Methods Using Matlab (2015), Michael McCourt
A quick note: While they’re still early in their ML/AI journey, Stack Overflow is a company that is well-known and beloved in technical communities, and often one of the first places we look to solve problems big or small. For that reason I’m excited to share the conversation with you. That said, if you’re looking for an in-depth technical analysis, you might want to skip this one. Back to the Basics: Introducing Stack Overflow & Prashanth Chandraseka Stack Overflow was founded in 2008, and has been growing massively ever since. The site hosts almost 100 million monthly visitors from around the world, who can access 50 million questions, which amounts to 1 TB of publicly accessible data in the form of questions, answers, and comments. While most people recognize Stack Overflow for its public-facing Q&A site, the company is also developing products to help businesses foster collaboration by using private versions of Stack Overflow-like sites within their organizations. Prashanth became CEO of Stack Overflow in October 2019. Having spearheaded Rackspace’s pivot from hosting to managing cloud services, Prashanth had firsthand experience in growing a business in this direction quickly. Impact of COVID at Stack Overflow Just 6 months after becoming Stack Overflow’s CEO, COVID-19 revolutionized the way the company was run. The impact was mixed. On the one hand, the widespread transition to a remote workforce increased the number of people coming to Stack Overflow. When Prashant joined the company, there were about 150,000 monthly signups on the public platform, a number which has since increased to the 200,000-250,000 range. As technology was one of the few sectors whose revenue increased over the pandemic, the transition to remote work actually also allowed for increased adoption of Stack Overflow’s Teams integration and advertising products. The objective of the Stack Overflow Teams product is to minimize context switching and conserve the energy of technologists. In order to do this, the team created an asynchronous collaboration tool that automates some frequent responses, using common tools like Slack and Microsoft Teams. For example: imagine a new employee asks you where a specific repository is via Slack. The Stack Overflow integration then suggests similar questions and answers that have already been documented. The new employee gets an immediate response, and you don’t have to spend time explaining or tracking down information. Since technology companies are always looking for quality places to showcase their products, Stack Overflow was the perfect place to reach technical communities. Stack Overflow’s advertising product Reach and Relevance saw a significant increase in advertising revenue during 2020. Despite these positives, the same negative things that affected the world in 2020 deeply affected the personnel at Stack Overflow. Personal health issues, families that were affected by COVID, and the cultural reckoning around racism and inclusion, made it a challenging time. There were additional challenges, such as having to pivot away from certain products, like a job listing product that was too unpredictable as a steady revenue source. Recent Acquisition by Prosus Stack Overflow was recently acquired by Prosus, a move that Prashanth attributes to the companies’ similar missions and alignment of their long-term goals. While Stack Overflow had been approached many times over the years, Prashanth believed Prosus could help Stack Overflow think big and grow to its full potential in a global, community-oriented way. “Prosus builds leading consumer internet companies and empowers people and enriches communities.” When you think about that statement, that speaks immediately to the heart and soul of this company. Stack Overflow Community Philosophy Stack Overflow has a few principles that guide the way they approach building and managing communities. 1. Focus on a shared identity. Their mission is guided by focusing on elements Stack Overflow users share, like a love of problem solving. 2. Create sufficient incentives for people to contribute and participate in the system. At Stack Overflow, that incentive is helping out your fellow developer, showcasing your knowledge, and being recognized for it. 3. Build with the community instead of for the community. Prashanth sees it as “common sense” to recognize the community members as equal partners, and the team has built in modes of feedback and collaboration through moderator mechanisms and an email newsletter. 4. Break down silos and build bridges across various communities. One way Prashanth decided to do this was by hiring Felipe Baudette (formerly of Reddit) as the new VP of Community. “Ultimately, it’s all about making sure that we’re building thriving communities.” One distinction Prashanth mentions is that while Stack Overflow is a community space where members share information, it is not a social network. It’s less a matter of sharing opinions, and more a matter of right and wrong answers: either the code will work or it won’t. He reiterates that people appreciate Stack Overflow because it gives them the information they need to solve their problems. Data at Stack Overflow Stack Overflow is relatively early on in their adoption of AI. While there’s a ton of interesting data they have collected (70 TB worth!), they are currently building out a team who will be able to do creative things with it. One of the most interesting data sources they collect is a developer survey which polls about a hundred thousand people; this past year, 85,000 people responded. While this yearly survey has a lot of great insights, Prashanth would love to be able to generate valuable insights on a more frequent basis. Prashanth’s favorite finding was that 70% of developers reported they learn a new technology at least once a year. While he knew that developers liked to learn new things, 70% was a much higher number than expected. Prashanth also mentioned that 60% of these developers are learning to code from online resources. While there is some amount of structured data, the vast majority of what the team collects is unstructured data in the form of Q&A text, comments, and tags. They have also developed a network graph that can track a person’s workflow and help inform how users engage with the platform. Stack Overflow is currently establishing the groundwork for a machine learning and data science team– with a data lake, training spaces, and other necessities for AI. While they are still early in their journey, there are already two interesting endeavors they’ve created. Early ML Applications: The Unfriendly Robot and Teams Integration One of these projects involves making Stack Overflow a more amenable place for beginners. A common criticism of Stack Overflow is that it’s a very harsh environment for a new user, as more experienced Stack Overflow users will shoot down novice questions if they see them as “bad”. The Unfriendly Robot is a bot that scans all the public exchanges and flags inappropriate comments to a question, noting any “unfriendly” language. The team is continuing to evaluate the best way to build a softer “landing zone”, a place where people can onboard in a friendly environment and try out a few things before they actually go and ask that first question. The other area where ML is being used is in a bot in the Stack Overflow Teams product, the same product that’s integrated with Slack and Microsoft Teams. The bot helps suggest answers to FAQs. When someone asks a question on the Stack Overflow for Teams, the bot can provide the user with a few other questions that sound similar and offer to pull up the answer that was documented previously. This allows for better remote onboarding and less time spent responding to the same questions. The Future of Stack Overflow Two of the questions that drive future innovation at Stack Overflow are: How do you improve efficiency and how do you remove context switching? Moving forward, Prashanth hopes to automate parts of the Stack Overflow workflow that make it even easier and more efficient. Another goal Prashanth has is expanding their US-centric customer base internationally, in partnership with Prosus. Prashanth also hopes to integrate Stack Overflow even more deeply into the technologist workflow, ideally creating “connective tissue” between that public platform and the private teams version, and between Stack Overflow instances and other developer tools like IDEs. Ultimately, Prashant wants to build thriving communities and effective remote workflows that embrace a culture of learning. He imagines Stack Overflow integrations helping technologists work and scale more effectively, ultimately helping to build better technology for the future of humanity. To hear more about what’s going on at Stack Overflow, you can listen to the full episode here!
Over the last few years, Deep Learning technologies have made a tremendous impact on Medical Imaging. In this session participants will hear about some of the latest advances in the following areas: medical image segmentation and classification and object detection in medical images. A review of these advances will be presented via a detailed analysis of research papers from prestigious journals such as Nature Methods. In-Depth Summary Details: Title: “Federated Learning: A privacy-preserving method for training deep learning models using healthcare data.” Speaker: Dr. Anthony Reina Federated learning is a distributed machine learning approach that enables organizations to collaborate without sharing sensitive data, such as, patient records, financial data, or classified secrets (McMahan, 2016; Sheller, Reina, Edwards, Martin, & Bakas, 2019; Yang, Liu, Chen, & Tong, 2019; Sheller et al., 2020). The basic premise behind federated learning is that the model moves to meet the data rather than the data moving to meet the model. Dr. Reina will discuss Intel’s work with the Federated Tumor Segmentation Initiative (FeTS) which uses Intel’s open-sourced federated learning framework across data from different hospitals to train an AI model for tumor segmentation from MRI scans of the brain. Title: Deep neural ensembles for improved pulmonary abnormality detection in chest radiographs Speaker: Sivaramakrishnan Rajaraman Cardiopulmonary diseases account for a significant proportion of deaths and disabilities across the world.Chest X-rays are a common diagnostic imaging modality for confirming intra-thoracic cardiopulmonary abnormalities. However, there remains an acute shortage of expert radiologists, particularly in under-resourced settings that results in interpretation delays and could have global health impact. These issues can be mitigated by an artificial intelligence (AI) powered computer-aided diagnostic (CADx) system.Such a system could help supplement decision-making and improve throughput while preserving and possibly improving the standard-of-care. A majority of such AI-based diagnostic tools at present use data-driven deep learning (DL) models that perform automated feature extraction and classification. Convolutional neural networks (CNN), a class of DL models, have gained significant research prominence in tasks related to image classification, detection, and localization. The literature reveals that they deliver promising results that scale impressively with an increasing number of training samples and computational resources. However, the techniques may be adversely impacted due to their sensitivity to high variance or fluctuations in training data. Ensemble learning helps mitigate these by combining predictions and blending intelligence from multiple learning algorithms. Complex non-linear functions constructed within ensembles help improve robustness and generalization. Empirical result predictions have demonstrated superiority over the conventional approach with stand-alone CNN models. In this talk, I will describe example work at the NLM that use model ensembles to improve pulmonary abnormality detection in chest radiographs.
To hear about Josh Tobin’s various projects in robot learning, check out the full interview! The discussion on projects outside of the NeurIPS paper picks up at 15:58 in the podcast. Enjoy! Robots have a particularly hard time with “reading a room.” In order to know how to act, they first need to understand their environments. It’s like learning where everything is when you enter a grocery store for the first time. For machines, processing the world in terms of three dimensional spatial awareness–figuring out where objects are, how they are positioned, and even the robot’s own placement in the environment–is incredibly complex. In the real-world, scenarios can be unpredictable and challenging to simulate, so how can we improve perception so machines can successfully understand the world around them? Josh Tobin has dedicated his research to improving robots’ ability to accomplish real-world tasks. He finished his PhD at UC Berkeley under Pieter Abbeel, who we had the opportunity to interview for a Reinforcement Learning Deep Dive. Josh recently took a break from his role at OpenAI and joins us from NeurIPS 2019, where he presented his paper on Geometry Aware Neural Rendering, where they successfully improve 3D modeling to more complex, higher dimensional scenes. Neural Rendering and Generative Query Networks Neural rendering is the practice of observing a scene from multiple viewpoints and having a neural network model render an image of that scene from a different, arbitrary viewpoint. For example, if you have three cameras taking in different perspectives of an environment, the model would use that information to reconstruct a fourth viewpoint. The intuition behind this process is that the model has demonstrated a good understanding of the space if it can predict a strong representation of that view. Tobin’s work is an extension of the ideas presented in a paper from DeepMind on Neural Scene Representation and Rendering. The paper introduces Generative Query Networks (GQN), which the DeepMind paper refers to as “a framework within which machines learn to represent scenes using only their own sensors.” The significance of GQNs is that they do not rely on a massive amount of human-labeled data to produce scene representations. Instead, the model gleans the “essentials” of a scene from images, “constructs an internal representation, and uses this to predict the appearance of that scene.” In GQN, they take the problem of neural rendering and set up a model structure that works with an encoder-decoder architecture. As Josh describes, “The encoder takes each of the viewpoints and maps them through a convolutional neural network independently, so you get a representation for each of those viewpoints. Those representations are summed.” This creates a representation for the entire scene which is then passed on to the decoder. “The decoder’s job is to…go through this multi-step process of turning [the representation] into what it thinks the image from that viewpoint should look like.” GQNs are a powerful development, but there is still a bottleneck that occurs when the representation produced by the encoder is passed to the decoder. This is where the geometry aware component (the main contribution of Tobin’s paper) comes in. Geometry Awareness: Attention Mechanism and Epipolar Geometry Josh’s primary goal “was to extend GQN to more complex, more realistic scenes” meaning, “higher-dimensional images, higher-dimensional robot morphologies, and more complex objects.” Their approach was to use a scaled dot-product attention mechanism. “The way the attention mechanism works is by taking advantage of this fact of 3D geometry called epipolar geometry,” which refers to viewing something from two points and defining the relationship between them. In this case, epipolar geometry refers to knowing “the geometry of the scene, so where the cameras are relative to one another.” If you’re a machine trying to render an image from a particular viewpoint, you want to “go back and look at all of the images that you’ve been given as context, and search over those images for relevant information. It turns out, if you use the geometry of the scene [epipolar geometry]… then you can constrain that search to a line in each of the contexts viewpoints” and attend to the pixels that are most relevant to the image you’re constructing. “For each pixel, we’re constructing this vector that represents a line. When you aggregate all of those you have two spatial dimensions for the image. So you get this 3D tensor and you’re dot-producting the image that you’re trying to render…and that’s the attention mechanism.” The New Data Sets In order to evaluate the performance of the new model they developed several new data sets: In-Hand OpenAI Block Manipulation. This is a precursor to Open AI’s Rubik’s Cube project. In this data set, “You have a bunch of cameras looking at a robot hand that’s holding a block. The colors of the cube, the background, and the hand are randomized and the hand and the cube can be in any pose.” Disco Humanoid. This is Josh’s term for the data set because it looks like a “humanoid model that’s doing crazy jumping-in-the-air dance moves.” It’s similar to the MuJoCo humanoid model except that the colors, poses of the joints, and the lighting are completely randomized. It’s meant to test “whether you can model robots that have complex internal states rates with this high-dimensional robot that you need to model in any pose.” Rooms-Random-Objects. The most challenging data set they introduced involved simulations of a room with objects taken from ShapeNet, a data set with over 51,000 3D models. “Each of the million scenes that we generated had a different set of objects placed in the scene. It’s really challenging because the model needs to understand how to render essentially any type of object.” Randomization is a key part in each data set. As Josh believes “If you want to train a model and simulation that generalizes the real world, one of the most effective ways of doing that is to massively randomize every aspect of the simulator.” Evaluating Performance and Results To evaluate their results, the team compared their model with GQN using several metrics, including the lower bound on the negative log likelihood (the ELBO), per-pixel mean absolute error (L1 and L2), and by qualitatively reviewing rendered images of actual scenes. “The main result of the paper is we introduced a few new data sets that capture some of those properties, and then we showed that our attention mechanism produces qualitatively and quantitatively much better results on new, more complex datasets.” They have yet to test the system in the real world. Josh confirms “right now, it only captures 3D structure, it doesn’t capture semantics, or dynamics, or anything like that, but I think it’s an early step along that path.” Josh is working on various related projects around data efficiency in reinforcement learning and some fascinating work on sim-to-real applications. To hear about them, check out the full interview! The discussion on projects outside of the NeurIPS paper picks up at 15:58 in the podcast. Enjoy!
Stefan Lee is involved in a number of projects around emergent communication. To hear about more of them, in addition to the ViLBERT model, check out the full interview! TWiML Talk #358 with Stefan Lee. One of the major barriers keeping robots from full immersion in our daily lives, is the reality that meaningful human interaction is dependent on the interpretation of visual and linguistic communication. Humans (and most living creatures) rely on these signals to contextualize, operate and organize our actions in relation to the world around us. Robots are extremely limited in their capacity to translate visual and linguistic inputs, which is why it remains challenging to hold smooth conversation with a robot. Ensuring that robots and humans can understand each other sufficiently is at the center of Stefan Lee’s research. Stefan is an assistant professor at Oregon State University for the Electrical Engineering and Computer Science Department. Stefan, along with his research team, held a number of talks and presentations at NeurIPS 2019. One highlight of their recent work is the development of a model called ViLBERT (Vision and Language BERT), published in their paper, ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks. BERT Models for Natural Language Processing BERT (Bidirectional Encoder Representations from Transformers) is Google’s popularized model that has revolutionized natural language processing (NLP). BERT is what’s called a language model, meaning it is trained to predict the future words in a sentence based on past words. Another example is GPT-2, which caused a controversy when released by OpenAI last year. Part of what makes them so special is that they are bidirectional, meaning they can contextualize language by reading data from both the left and right. This builds relationships between words and helps the model make more informed predictions about related words. They also pre-train models on large sets of unlabeled data using a transformer architecture, so data sequences don’t need to be processed in order, enabling parallel computations. Models like BERT work by taking “a large language corpus and they learn certain little things to build supervision from unlabeled data. They’ll mask out a few words and have them re-predict it based on the other linguistic context, or they’ll ask if a sentence follows another sentence in text.” Extending BERT to the Visual Domain ViLBERT is an extension of the BERT technique. To apply the BERT model to vision and language, the team worked with a data set called Conceptual Captions, composed of around 3 million images paired with alt-text. Their method is to mask out random parts of the image, and then ask the model to reconstruct the rest of the image given the associated alt-text. “Likewise, we’re asking, does this sentence match with this image or not? or masking out parts of the language and having it reconstruct from the image and the text. We’re designing this self-supervised multi-model task with this large weekly supervised data source.” Visual Grounding Stefan describes that “Most of natural language processing is just learning based on its association with other words. Likewise on the visual side, you’re learning to represent some sparse set of classes. Those classes often relate to specific nouns, but they don’t have a sense of closeness, so there’s no idea that the feature for a cat should be close to the feature for a tiger…The point of ViLBERT is to try to learn these associations between vision and language directly. This is something we usually call visual grounding of a word.” Humans do this naturally because we often have the inherent context to imagine a visual representation of something. For example, the words “wolf head” might bring up certain imagery of wolves, but machines lack the same visual associations. What Stephan is working towards with ViLBERT is to present the agent with something like, “red ball” and have it interpret that to reconstruct an image of a red ball. How Vilbert Works: Bounding Boxes and Co-Attentional Transformer In object detection, bounding boxes are used to describe the target area of the object being observed. For the purpose of ViLBERT, an image is dissected as a set of bounding boxes that are independent of each other. Each box is given a positional encoding that shows where the box was pulled from, but ultimately the order of the sequence is not important. BERT works in the same way where you have a sequence of words (a sentence) that are treated as independent inputs and given a positional embedding. “It’s just a set. It’s an unordered set. In fact, the actual input API for BERT looks the same in our model for the visual side and the linguistic side.” The bounding boxes are output by an R-CNN model trained on the Visual Genome. The R-CNN model “can produce quite a lot of bounding boxes and you can sample from it if you’d like to increase the randomness.” Something to note is that many of the bounding boxes are not well aligned and the data could come back as fairly noisy. “Sometimes you’ll have an object like a road…it doesn’t do a great job of honing in on specific objects.” While the model is not trained on the visuals from scratch, it still has to learn the association even when it might not be obvious. To train the model, certain parts of the image (bounding boxes) are removed, and the model is asked about alignment between the alt-text and the image. The distinction between BERT is that “In BERT, it’s a sentence and the next sentence, and you’re predicting whether one comes after the other, but in our case, it’s an image and a sentence, and we’re asking does this align or not? Is this actually a pair from conceptual captions?” At the end of the process, “what we get is a model that has built some representations that bridge between vision and language” which can then be fine-tuned to fit a variety of other tasks. Applications and Limitations Since the ViLBERT paper, the model has been applied to around a dozen different vision and language tasks. “You can pre-train this and then use it as a base to perform fairly well, fairly quickly, on a wide range of visual and language reasoning tasks.” In addition to fine-tuning for specific task types, you can adjust for specific types of data or data language relationships. One useful adaptation is for Visual Question and Answering (VQA) to help those who are visually impaired ask questions about the world around them, and receive descriptive answers in return. To modify the model for VQA, you could feed the questions as the text inputs and “train an output that predicts my subset of answers.” ViLBERT is pre-trained on a dataset of images and captions as the text input. For VQA, you would use the questions as the text input and have the model reconstruct answers as the output. While ViLBERT is a solid starting point for the field, Stefan notes that the grounding component of the research is still underdeveloped. For example, if the model is trained for VQA on a limited dataset like COCO images, there may be objects that are not accounted for because the machine never learned they existed. “One example that I like to show from a recent paper is that [COCO images] don’t have any guns. If we’ve fed this caption trained on COCO with an image of a man in a red hat with a red shirt holding a shotgun, and the caption is a man in a red hat and a red shirt holding a baseball bat, because he’s wearing what looks like a baseball uniform and he’s got something in his hands. It might as well be a baseball bat. If we talk back to these potential applications of helping people with visual impairment, that kind of mistake doesn’t seem justifiable.” Future Directions For Visual Grounding One related area of research that Stefan has started to branch into is the interpretation of motion. The problem with images is that it can often be difficult to distinguish between active behaviors and stationary behaviors. “For a long time in the community, grounding has been on static images, but there’s actually a lot of concepts that rely on motion, that rely on interaction to ground. I could give you a photo and you could tell me it looks like people are talking, but they could just be sitting there quietly as well.” There is less emphasis on the interaction, which is a key element not only to understanding communication, but for accuracy in reading a social situation. Machines are not yet able to catch on to these distinctions and it’s a growing area of interest for Stefan. For more on what Stefan is up to, be sure to check out the full interview with Stefan Lee for a deeper understanding of ViLBERT and the numerous projects the team is working on.
We're excited to share our third annual Black in AI series! When you're done with this year's series, make sure you check out the previous BAI series: Black in AI 2019 - Black in AI 2018.
How does LinkedIn allow its data scientists to access aggregate user data for exploratory analytics while maintaining its users' privacy? That was the question at the heart of our recent conversation with Ryan Rogers, a senior software engineer in data science at the company. The answer, it turns out, is through differential privacy, a topic we've covered here on the show quite extensively over the years. Differential privacy is a system for publicly sharing information about a dataset by describing patterns of groups within the dataset, the catch is you have to do this without revealing information about individuals in the dataset (privacy). Ryan currently applies differential privacy at LinkedIn, but he has worked in the field, and on the related topic of federated learning, for quite some time. He was introduced to the subject as a PhD student at the University of Pennsylvania, where he worked closely with Aaron Roth, who we had the pleasure of interviewing back in 2018. Ryan later worked at Apple, where he focused on the local model of differential privacy, meaning differential privacy is performed on individual users' local devices before being collected for analysis. (Apple uses this, for example, to better understand our favorite emojis 🤯 👍👏). Not surprisingly, they do things a bit differently at LinkedIn. They utilize a central model, where the user's actual data is stored in a central database, with differential privacy applied before the data is made available for analysis. (Another interesting use case that Ryan mentioned in the interview: the U.S. Census Bureau has announced plans to publish 2020 census data using differential privacy.) Ryan recently put together a research paper with his LinkedIn colleague, David Durfee, that they presented as a spotlight talk at NeurIPS in Vancouver. The title of the paper is a bit daunting, but we break it down in the interview. You can check out the paper here: Practical Differentially Private Top-k Selection with Pay-what-you-get Composition. There are two major components to the paper. First, they wanted to offer practical algorithms that you can layer on top of existing systems to achieve differential privacy for a very common type of query: the "Top-k" query, which means helping answer questions like "what are the top 10 articles that members are engaging with across LinkedIn?" Secondly, because privacy is reduced when users are allowed to make multiple queries of a differentially private system, Ryan's team developed an innovative way to ensure that their systems accurately account for the information the system returns to users over the course of a session. It's called Pay-what-you-get Composition. One of the big innovations of the paper is discovering the connection between a common algorithm for implementing differential privacy, the exponential mechanism, and Gumbel noise, which is commonly used in machine learning. One of the really nice connections that we made in our paper was that actually the exponential mechanism can be implemented by adding something called Gumbel noise, rather than Laplace noise. Gumbel noise actually pops up in machine learning. It's something that you would do to report the category that has the highest weight, [using what is] called the Gumbel Max Noise Trick. It turned out that we could use that with the exponential mechanism to get a differentially private algorithm. [...] Typically, to solve top-k, you would use the exponential mechanism k different times⁠ —you can now do this in one shot by just adding Gumbel noise to [existing algorithms] and report the k values that are in the the top […]which made it a lot more efficient and practical. When asked what he was most excited about for the future of differential privacy Ryan cited the progress in open source projects. This is the future of private data analytics. It's really important to be transparent with how you're doing things, otherwise if you're just touting that you're private and you're not revealing what it is, then is it really private? He pointed out the open-source collaboration between Microsoft and Harvard's Institute for Quantitative Social Sciences. The project aims to create an open-source platform that allows researchers to share datasets containing personal information while preserving the privacy of individuals. Ryan expects such efforts to bring more people to the field, encouraging applications of differential privacy that work in practice and at scale. Listen to the interview with Ryan to get the full scope! And if you want to go deeper into differential privacy check out our series of interviews on the topic from 2018. Thanks to LinkedIn for sponsoring today's show! LinkedIn Engineering solves complex problems at scale to create economic opportunity for every member of the global workforce. AI and ML are integral aspects of almost every product the company builds for its members and customers. LinkedIn's highly structured dataset gives their data scientists and researchers the ability to conduct applied research to improve member experiences. To learn more about the work of LinkedIn Engineering, please visit engineering.linkedin.com/blog.
This course works through the Stanford CS224 Winter 2019 course lectures, and discusses additional resources for better understanding. This study group is for anyone who is interested in NLP, including beginners! We try to discuss the lectures in as much detail as possible. Experience with working in Python and Neural Networks is helpful. Meetups take place every Saturday at 8am PT and will run through 11/21/2020. A detailed weekly schedule can be found here. Resources for the course can be found here: Course home page, Winter 2019 Video Playlist Final Project Reports, Winter 2019 You can also find video recordings of the study group sessions here. The playlist is updated as the course progresses.
Another #TWIMLcon short with the wonderful Rosie Pongracz and Trisha Mahoney, from a Founding sponsor who you all know, IBM. Rosie is the  World Wide Director of Technical Go-to-Market and Evangelism for Data Science and Trisha is a Senior Tech Evangelist. We chat about the latest IBM research, projects, and products, including AI Fairness 360, which will be the focus of Tricia’s session at TWIMLcon. The IBM booth also promises to bring the heat, with a variety of open source projects and resources for the data science community. See you there! Sam Charrington: [00:00:00] All right everyone. I've got Rosie Pongracz and Trisha Mahoney from IBM on. Rosie is the Worldwide Director of Technical go-to-market and Evangelism for Data Science and AI and Trisha is a Senior Tech Evangelist in Machine Learning & AI and they are both instrumental in IBM's support for the TWIMLcon: AI Platforms conference. Rosie and Trisha, it's so exciting to be able to talk to you. Rosie Pongracz: [00:00:27] We are excited to be here, Sam! So happy to be a supporter of TWIMLcon and all the great work you do. Trisha Mahoney: [00:00:33] Thanks for having us, Sam. Sam Charrington: [00:00:34] Absolutely. Thank you. So, I don't know if it makes sense to say who is IBM? [laughs] You know in this context, I think most people who hear this know what IBM is but, you know, maybe you can talk a little bit about the company's involvement in the AI Platform space and, you know, why you're, you know what really kind of created the interest in supporting this, this conference. Rosie Pongracz: [00:01:00] Absolutely. So, yes, I would imagine most of the listeners already know IBM. We are long-standing, I'd say, evangelist, product producer, supporters of open source, anything for AI. And I'd say most of the current recognition goes back to Watson, of course, and the Jeopardy challenge.   But from that, IBM has evolved...what was that, almost ten years ago, to create some significant products. Not only have we made our way to the cloud should I say, and supports hybrid clouds for our clients and bringing them through the digital transformation, but we also have a good range of, of tools that help people not only do data science and machine learning but also scaled those, operationalized those, and bring them to production. I think if anything, IBM is known for its expertise in enterprise-scale and wide range of industry solutions. And that's really what we're doing. We're involved in open source. So quite a few open-source projects that are AI and data science and ML related, as well as products that can help our clients bring that AI to their business.  Sam Charrington: [00:02:16] Awesome. And I know that I've covered some of those products in our recent e-books in the platform space. Both the Fabric for Deep Learning open source project, which I talked about in our Kubernetes for ML and DL e-book, as well as the Watson Studio products which I believe came up in the ML Platforms e-book. Are there other products that IBM is kind of focused on in this space? Rosie Pongracz: [00:02:43] I think you captured the main ones. Especially the ones my team has been involved in. There's Watson Studio, Watson Machine Learning, Watson Open Scale. And if you look at Studio, it's more or less it's an IDE of sorts for data scientists, built on Jupiter Notebooks. ML, uh Watson ML is for running those machine learning algorithms. And then Watson Open Scale is for at scale.  And actually one of the big pieces of that pipeline, if you look at all those pieces along the pipeline or the platform, if you will is one of the areas that Trisha's going to be talking about which is the AI fairness and bias, which is a really important piece of the pipeline that we're proud to be incorporating.  I think you caught all the products. There's a significant amount of open-source that we're also involved in and, like I said, bringing those into our products and also supporting those communities like the Jupiter community, like the Linux Foundation, AI. Those are also very important projects and places where IBM has been involved as well.  Sam Charrington: [00:03:53] That's right. We recently did a podcast with Luciano Resende, who is at IBM and works on the Jupiter Enterprise Hub project, I believe is the name of it? Rosie Pongracz: [00:04:03] Yup. Jupiter Enterprise Gateway is correct. Yes. Sam Charrington: [00:04:05] Got it. Jupiter Enterprise Gateway. Rosie Pongracz: [00:04:07] Yeah. Sam Charrington: [00:04:08] So in addition to all of the products and open-source that you're working on in this space, you're also out there evangelizing the whole idea of ML Ops. You ran a workshop on this topic at the OzCon Conference recently. Maybe talk a little bit about your perspective on ML Ops and why that's so interesting to you. Rosie Pongracz: [00:04:29] Yeah. I think it goes back to where, where IBM can really make a difference is that we have, we'll have literally hundreds of years, decades, of experience in helping our enterprise clients do things at scale. And that is across industry. So if you look at all of the products that we have and you also look at something like cloud pak for data, which is bringing those containerized applications to any cloud, really, it is about giving our clients flexibility, helping them modernize. It's helping do things at scale.   Now a lot of our clients also have businesses that they're trying to transform so when you talk about ML Ops, certainly, you look at data science, I kind of look at that akin to a desktop where a developer works on. It's great to be able to develop those algorithms on your desktop and test that out on data sets, but when you really want to implement it, you're talking there's a whole kind of dev-ops cycle, if you will, applying that to AI and then machine learning.   And IBM has been there with its clients in the early days of Java. It's been there in the early days of cloud. And we're also taking that now into kind of the next realm if you will, the next era of bringing AI to businesses at scale. So how do you take your current applications and embed AI in those? Or how are you creating new ways to use your data and to modernize your business? And IBM you know, it's just near and dear to our client's heart. It's near and dear to who we are as a company in being able to do things at scale. And you have to have a platform. You have to have a way to operationalize that. It's great to run little science experiments to try things out and test things and fail fast, but when you start to operationalize, that's where the ML at scale, ML Ops, is really going to start to be important. Sam Charrington: [00:06:25] Mm-hmm [affirmative]. I was at the last IBM Think Conference, which is its big user conference and had an opportunity to hear Rob Thomas talk about, you know, one of the key things that he sees as being a determinant of enterprises finding success in machine learning and AI is the number of experiments that they're able to run and being able to scale that so that they can run those experiments en masse. Rosie Pongracz: [00:06:51] Yeah absolutely. That's an important piece of what IBM is helping enable our clients to do. And with our products that is definitely what we're striving for. You've got to be able to experiment. And then when you do want to operationalize, you got to be able to do that at scale.   Some of the clients we work with have some of the biggest applications running for their enterprise for their customers. And they depend on IBM to do that. So how do we bring that into, you know, this experimentation mode? Because you're absolutely right. Now it's not, you know, much more in...it's not about, you know, building one app and then releasing that. It's, as you know, the world is very much agile, you've got to fail fast. You've got to experiment. You've got to understand.   And with data science, that is absolutely sort of the MO. That's sort of the way you operate; is how do you, how do you know what works? And then if, when you...you know, you also have to retrain. So there's a lot of differences to building AI and building data science in a [inaudible] scale that is slightly different than just building applications if you will. Sam Charrington: [00:07:55] Mm-hmm [affirmative]. Mm-hmm [affirmative]. So, Trisha, you're going be speaking at the conference. Tell us a little bit about your topic and what attendees can expect when they come to your session. Trisha Mahoney: [00:08:06] Right. So, I'm going to be speaking on AI Fairness 360. And this is a comprehensive toolkit created by IBM researchers. And what we focus on is detecting and understanding and mitigating unwanted machine learning bias. So the toolkit is open source. It's in Python and it contains over 75 fairness metrics, ten bias mitigation algorithms, and fairness metrics with explanations. So one of the key components to this is that it has some of the most cutting edge metrics and algorithms across academia and industry today. So it's not just an IBM thing, it includes the algorithms from researchers from Google, Stanford, Cornell. That's just a few.   But what it really focuses on is teaching people how to learn to measure bias in their data sets and models. And how to apply fairness algorithms throughout the pipeline. So you know the big focus is on data science leaders, practitioners, and also legal and ethic stakeholders who would be a part of this.  So, just a few things that I'll go through in the talk is when you would apply pre-processing algorithms to manipulate your training data, in- processing algorithms, for incorporating fairness into your training algorithms itself, as well as post-processing, de-biasing algorithms. And, you know, one of the key things we wanted to get across is, I'm working on an O'Reilly book on AI fairness and bias with our researchers. So, you know, the key thing is that you know, this is a problem we think may prevent AI from reaching its full potential if we can't remove bias.   So, the thing we want to get across is that this is a long data science initiative. If you want to remove bias throughout your pipeline, so it involves a lot of stakeholders in your company, and that it can be very complex. So the way you define fairness and bias leads down into the types of metrics and algorithms you use. So, you know there are a lot of complexities. And the hope is that data science teams need to work with people throughout their org; they can't really make these decisions on their own, as they may actually break the law in some cases with their algorithms.   So, you know, I'll go into in the short period of time, kind of some of the trade-offs that data science teams have to make between model accuracy and removing bias, and talk about what they do for acceptable thresholds for each.   And the last thing on the ML Ops piece is I'll also do a demo in Watson Open Scale. And this is where you, you know, have models in production and you need to detect and remove bias from models, you know that are, aren't in an experimentation environment, right? So within Watson Open Scale, you can automatically detect fairness, issues at run time. And we essentially just do this by comparing the difference between rates at which different groups receive the same outcomes.  So are different minority groups, or men or women being approved for loans at the same time. So that's just an example. So that's kind of the top things that I'll go through on the toolkit and, I've heard many people say that others do bias talks on the problem that we have. But AI Fairness 360 is one of the few that's bringing a solution to the table on how to fix this within the machine learning pipeline.  Sam Charrington: [00:11:29] Yeah, I think that's one of the most exciting things about the talk from our perspective is that it's not just talking about the challenges that exist, but also how to integrate a concrete toolkit into your pipeline. And whether it's Fairness 360 or something else, but how to, integrate tools into your pipeline so that you can detect and mitigate bias, just very concretely as opposed to talking about it abstractly. Trisha Mahoney: [00:11:58] Correct. And I think the bridge that this creates is, you know, there are a lot of new fairness research techniques out there, but this toolkit sort of gets them into production and accessible in a way that data scientists can use. So, I think this is considered the most comprehensive toolkit to do that on the market today. Sam Charrington: [00:12:18] Mm-hmm [affirmative]. So Rosie in addition to Trisha's session, you'll also be exhibiting at the conference in our community hall. What can attendees expect to see at the IBM booth there? Rosie Pongracz: [00:12:30] Yeah, we're excited to be there too. So you'll see several things. We are going to be talking about the relevant open source projects like AI Fairness 360 that Trisha mentioned and also AI Explainability 360, which is another new toolkit. And we have, actually, a whole host of, projects that I won't go into here, but we can talk through those and see where IBM is contributed and working on open source projects like the Jupiter Enterprise Gateway that you mentioned as well.   They'll also see our, our products, and how those work together in helping operationalize and bring AI platforms to reality. And we'll also be talking about our data science community, which is a place that not only can product users go and share and collaborate, but also we have some great technical solution type content on there, with the goal of that being that IBM has a lot of deep rich solutions that we're building. As I mentioned earlier, industry-specific, or transformation type of projects and those are the types of materials that we're building there.  We've heard many people, both academic and industry, say it's great to talk about all this theoretical AI and what we'd really like to see is how are people putting that to work and solutions. So that's something that we're trying to bring to life on the community with many of [our] IBM experts all across any of our implementation folks, to our research folks. Sam Charrington: [00:14:01] Fantastic. Fantastic. Well, I'm really looking forward to seeing both of you at the event. And I am very gracious for your and IBM's support of the conference. Rosie Pongracz: [00:14:14] We are really excited to support what you're doing, Sam. I know you and I have worked together for many years through some technology transitions, so this is really appropriate and fun and fitting that we get to work together on something as exciting as what you're doing at TWIMLcon. Sam Charrington: [00:14:29] Absolutely. Thank you both. Rosie Pongracz: [00:14:31] Thank you. TWIMLcon: AI Platforms will be held on October 1st and 2nd at the Mission Bay Conference Center in San Francisco. Click here to learn more  
I had the chance to sit down with Scott Clark, Founder & CEO of SigOpt, a Founding sponsor of the upcoming TWIMLcon: AI Platforms! Scott discusses what SigOpt has been up to, the unique value he sees #TWIMLcon bringing to the ML/AI industry and what you can expect from the expert-driven SigOpt session and booth! Sam Charrington: [00:00:00] All right everyone, I am excited to have Scott Clark, founder and CEO of SigOpt. If you know Scott's name, it's because he's one of the few who has been on the TWIML AI podcast multiple times. Scott, welcome once again. Scott Clark: [00:00:13] Thanks, Sam. Always a pleasure to chat. Sam Charrington: [00:00:16] For those who haven't heard one of the previous episodes, why don't we get started by having you give us a really brief overview of your background. Scott Clark: [00:00:23] Yep. So I did my PhD in optimization in applied mathematics at Cornell. Spent a couple years at Yelp on their advertising team, helping them tune the various aspects of it, and working on releasing things like the Yelp academic dataset challenge. And then about five years ago, started SigOpt. Sam Charrington: [00:00:42] And so what is SigOpt? Scott Clark: [00:00:45] We're a software company. We build tools to help people build better models. We do this via an experimentation and optimization platform that bolts on top of any model or AI platform, allowing people to tune and tweak all the various configuration parameters of their models better, faster, and cheaper than alternative methods. We do that today with asset managers with over $300 billion of combined asset center management, Fortune 500 firms with $500 billion of total market cap, several dozen universities and research institutes around the world, as well as the US intelligence community, and many many more. Basically, anyone who has a model, we help them configure it and experiment with it to get it to the best performance. Sam Charrington: [00:01:29] So, Scott, SigOpt, and you personally as well, have been huge supporters of everything that we've done here at the podcast in terms of AI platforms, from the e-book that we recently published, The Definitive Guide to Machine Learning Platforms, to the upcoming conference, TWIMLcon: AI Platforms. Tell us a little bit about why you're so excited about the conference, TWIMLcon: AI Platforms, and the space in general around machine learning platforms. Scott Clark: [00:02:03] Definitely. We're super excited about this because we have the privilege of working with some of the most advanced firms in the world when it comes to AI and ML, and we've noticed that a lot of them have started to build these platforms over the last few years. As you start to productionalize AI, as you start to solve some of the low hanging fruit problems, you start to notice areas of overlap. Areas where engineers can build tools to make the entire modeling process a little bit more efficient, a little bit more scalable, a little bit more robust, et cetera. So a lot of our customers have been building these over the last few years, and SigOpt is kind of a seamless component, via our rest API, to bolt into them and help supercharge that experimentation and configuration tuning aspect of modeling.  So we're very excited to have someone be shining a light on the problem, and helping those firms that may not have been doing modeling and production for the last decade kind of get a leg up and skip over some of the trials and tribulations that those that went before them have already solved. Sam Charrington: [00:03:04] That's awesome. And so you're personally going to be delivering a session at the conference. What can attendees expect to get out of your session? Scott Clark: [00:03:15] Yeah. I'll be building upon some of the stuff in your ebook, talking about how people can make different trade-offs as they look to standardize various components of their machine learning platforms. How they think about what things need to be bespoke and built for their very specific use cases, and other things that can be standardized. Whether that's using open source tools or partnering with firms like SigOpt. I'll talk about the different trade-offs there, and how you can have standardization without necessarily constraining what your researchers and modelers are doing as well. Sam Charrington: [00:03:51] Yeah, I'm really glad that you're going to be talking about that because I think one of the things that I try to convey in the e-book is that there's really no one size fits all. Everyone's coming from a different technology legacy, solving a different set of problems, have a different set of skill sets, and it is important that everyone that starts off on this journey or, you know, is just taking the next step on their journey to figure out, you know, what are the things that make the most sense for them, and how to evaluate the increasing number of options that are available in this space, from open source to commercial products, to as-a-service offerings.   It sounds like you get a pretty broad view of that across the different customers that you get a chance to work with.  Scott Clark: [00:04:38] Definitely, and we see that every customer has a unique, domain they're working in, a unique set of problems, a unique context that they need to be aware of, and what might work well for market making high frequency trading firm is fundamentally different than an oil and gas company, which is fundamentally different than a credit card company. And by making sure that you leverage your expertise where- ... it can make a difference, and then use the best tools in the world where it's an orthogonal approach can really allow you to accelerate and amplify what you can do individually with constrained resources. Sam Charrington: [00:05:15] And so your session is called Exploring Trade-Offs and Experiment Management as Part of AI Platforms, and it'll be on Tuesday at 10:50 on our technology track. SigOpt is also going to have a presence in our community hall. Can you tell us a little bit about what folks can expect to find when they show up to your booth? Scott Clark: [00:05:37] Definitely. We'll have a handful of experts from our team on site walking through all the different lessons we've learned from working with these leading firms for many years now. We'll talk about different trade-offs they found, different pitfalls they found, and how they leverage experimentation to really empower their experts to get the most out of their modeling. Sam Charrington: [00:05:57] Awesome. Awesome. Well, I'm really looking forward to seeing you and the team at TWIMLcon, and I really can't express enough my thanks to you and the company for supporting the conference. Really, really excited about this. Scott Clark: [00:06:11] Likewise, Sam. It's always a pleasure to chat, and we're really looking forward to seeing you and this amazing conference that you put together. Sam Charrington: [00:06:17] Awesome. Thanks, Scott. Scott Clark: [00:06:19] Cheers.  TWIMLcon: AI Platforms will be held on October 1st and 2nd at the Mission Bay Conference Center in San Francisco. Click here to learn more  
Welcome to #TWIMLcon Shorts - a series where I sit down with some of our awesome Founding Sponsors and talk about their ML/AI journey, current work in the field and what we can expect from them at TWIMLcon: AI Platforms! First up is Luke Marsden, Founder & CEO of Dotscience. Based in Bristol, UK, Luke joins me to share the Dotscience story and why he is most excited for #TWIMLcon next month! From a stellar breakout session featuring the Dotscience manifesto to live demos at their booth, we can’t wait! Sam Charrington: [00:00:00] All right everyone, I am on the line with Luke Marsden. Luke is the founder and CEO of Dotscience a founding sponsor for TWIMLcon: AI Platforms. So Luke, we go back a little bit from your involvement in the docker space. I remember introducing you at a session at Dockercon quite a few years back, but for those who aren't familiar with your background, who are you? Luke Marsden: [00:00:51] So hey Sam, and thanks for having me on. My name is Luke Marsden, I'm the founder and CEO of Dotscience and I come from a devops background. My last startup was called Cluster HQ, and we were solving the problem of running stateful containers in docker. And so I'm a sort of serial entrepreneur based out of the UK. I live in the beautiful city of Bristol in the southwest and very excited to be involved with TWIML. Sam Charrington: [00:01:28] Awesome. So tell us a little bit about Dotscience and what the company is up to in the AI platform space.  Luke Marsden: [00:01:36] Yeah, sure. So we started Dotscience a couple of years ago. Initially, we were targeting the area of data versioning and devops but we quickly realized that the tool that we built which is an open source project called dotmesh was actually much more relevant and important to the world of AI and machine learning which has a big data versioning and reproducibility problems. So we pivoted to that about a year in, and we've been building an AI platform around that core concept of data versioning.  Sam Charrington: [00:02:13] So tell me a little bit more about that. How are you taking on data versioning? And why is that an important element of the puzzle for folks that are doing AI? Luke Marsden: [00:02:25] Absolutely. So there's really sort of four main pieces of the puzzle that I believe need to be solved to achieve devops for AI, devops for machine learning, and number one is reproducibility - and that's where the data versioning piece comes in. So what we've seen is that there's a lot of chaos and pain that happens when AI or ML teams start trying to operationalize the models that they're developing. And one of the big pain points is if you can't actually get back to the exact version of the data that you use to train your model, then you can't go back and solve problems with it. You can't fix bugs in the model or or really reliably understand sort of exactly where that model came from. So that's kind of that fundamental problem of like which version of the data that I trained is this model on and that's what we solve with with Dotscience. Every time you train a model in Dotscience, you are automatically versioning all of the dependent data sets that that model training happens on. And by using copy-on-write technology, which is a file system technology and in dotmesh, which is part of the Dotscience platform, it does that very efficiently using no more disk space than is required to achieve reproducibility. Sam Charrington: [00:03:52] Awesome. So tell me why are you excited about TWIMLcon: AI Platforms? Luke Marsden: [00:03:59] TWIMLcon looks to be an awesome event. We were actually planning on hosting our own event around the same time in San Francisco to promote Dotscience, but TWIML was such a good fit for what we're trying to do, and the themes and the topics that are being discussed in the space, that we decided to join forces with you guys and become a Founding sponsor rather than running our own things. So yeah, really, really excited and looking forward to it.  Sam Charrington: [00:04:34] That's fantastic and we are super appreciative to have you on board as a Founding sponsor, it is great to have your support in that way. When folks come to your breakout session at TWIMLcon, tell us a little bit about what you'll be covering there, who will be presenting, what can attendees expect to learn from the breakout session. Luke Marsden: [00:04:57] Yes, so the session will be run by my colleague Nick who's our principal data scientist, and the basic premise of the talk really touches on some of the things I mentioned earlier. There's a lot of chaos and pain trying to operationalize AI and that we have this manifesto of things that we believe are needed to go from, sort of the "no-process" process that is the default. So when you start an AI or machine learning project and you have maybe a small number of data scientists or machine learning engineers doing that work, they'll invent a process, right? Any technical group that's doing technical work will make up a process as they go based on the tools that they're familiar with and they'll do their best. But the point of the talk is that the "no-process process," it gets your first model into production when your team is small, but that's really where the problems begin and (Laughter) you end up with this sort of this kind of mess of models and data sets and deployments and hyperparameters and metrics and all these different things flying around, because machine learning is fundamentally more complicated than software development software engineering. And so, by just sort of doing things in an ad-hoc way, you get yourself into this sort of mess quite quickly, and this is something we've seen across hundreds of companies that we've spoken to in the industry. And so basically what we're proposing is a Manifesto, that you should make your machine learning process, the whole process of building, training, deploying, monitoring machine learning models that you should make that that whole process reproducible, accountable, collaborative, and continuous.  And so what I mean by reproducible is that somebody else should be able to come and reproduce the model that I trained now, like 9 or 12 months later without me still needing to be there, without me needing to have kept meticulous manual documentation. Somebody else should be able to go and rerun that model training against the same version of the data with the same version of Tensorflow with the same code, with the same hyperparameters, and get the same accuracy score to within a few percent. If your development environment isn't reproducible, then you won't be able to do that, but we believe that that is key to achieving devops for ML.  So anyway, that's kind of a snapshot of some of the things we'll be talking about in the session. So yeah, please please come along.   Sam Charrington: [00:08:00] Awesome. You'll also be present in TWIMLcon's Community Hall, what can attendees expect to see at the company's booth? Will they be able to get hands on?  Luke Marsden: [00:08:15] Absolutely, so we'll have live demos at the booth. You can see the full end-to-end platform and our Engineers as I speak in the early part of September today, are busily working on the latest features that we're going to have ready in time for the conference in true startup conference driven development mode. (Laughter) So, we will have the deploy to production and statistical monitoring pieces ready in time for the conference. So, it's probably going to be the first time that you can come and see those pieces of the product and and get hands-on with the product will be at TWIML, so please come and check it out. Sam Charrington: [00:09:00] Fantastic.  Luke, thanks so much for chatting with me about what you're up to and what you'll be showing at the event, we are super excited to have you on board with us for TWIMLcon: AI Platforms.   Luke Marsden: [00:09:10] Awesome. Thank you Sam. TWIMLcon: AI Platforms will be held on October 1st and 2nd at the Mission Bay Conference Center in San Francisco. Click here to learn more
We’ve got an amazing TWIMLcon agenda taking shape for you! We’ll be announcing speakers and more agenda details shortly, but in the meantime I wanted to share a preview of some of the keynote interviews we’ve got planned. I’m super pumped about them! TWIMLcon: AI Platforms will feature live “keynote interview” podcast recordings with accomplished guests in the industry. Read on to learn about a few of the ones we just announced: Andrew Ng. I can’t think of anyone who’s done more to bring new practitioners into the fields of machine learning and deep learning than Andrew and we’re so excited to be opening the event with him. He’ll be sharing what he’s learned helping many businesses with machine learning and AI, and also speak with us about where he sees the field going.     Hussein Mehanna. Hussein is the Head of AI/ML Cruise, the self-driving car company. Before Cruise, Hussein helped build Google’s Cloud ML Platform and Facebook’s FBLearner platform. He’ll be sharing some of the lessons he’s learned building ML platforms from scratch at some of the most advanced companies in the space, and applying these lessons with much smaller teams.   Fran Bell. Fran is the director of data science responsible for Data Science Platforms at Uber. Fran leads a team building use-case focused ML platforms supporting areas like Forecasting, Anomaly Detection, Segmentation, NLP & Conversational AI, and more. Her platforms sit on top of Uber’s Michelangelo, putting her in a unique position to speak with us about how both low-level and higher-level ML platforms can drive data scientist and developer productivity.   Beyond keynote interviews like these, we’ve got a bunch of interesting speakers lined up to share their successes and failures helping their organizations build and productionalize ML and deep learning models. Stay tuned for more details! TWIMLcon: AI Platforms will be held on October 1st and 2nd at the Mission Bay Conference Center in San Francisco. Click here to learn more  
Bits & Bytes Google scraps controversial AI ethics council days after it was announced. Google has shuttered its new artificial intelligence ethics council, a little more than a week after announcing it, in large part due to employee reactions to the appointment of a conservative think tank leader to the group. WIPO launches AI-based image search tool for brands. The World Intellectual Property Organization (WIPO) has launched a new AI-powered image search technology that uses deep learning to identify combinations of concepts within an image, thus making it faster and easier to establish a trademark’s distinctiveness. Tim Apple poaches Ian GANfellow. Apple has poached another of Google’s top AI researchers. This time it’s Ian Goodfellow, best known as the creator of GANs, or generative adversarial networks, who has joined Apple in a director role. FDA Proposes Regulatory Framework for AI- and Machine Learning-Driven SaMD. US Food and Drug Administration (FDA) requested feedback on a new discussion paper that proposes applying a “focused review” approach to premarket assessments of software as a medical device (SaMD) technologies that are powered by AI and ML. [Paper] Qualcomm Reveals “Cloud AI 100” AI Inference Accelerator. Qualcomm announced their first discrete dedicated AI processors, the Qualcomm Cloud AI 100 family. The chip, which it expects to begin producing in 2020, is designed for use in datacenters to meet increasing demand for AI inference processing. Google launches an end-to-end AI platform. At Google Next ’19, Google announced the beta version of its end-to-end AI platform, which aims to offer developers and data scientists a one-stop shop for building, testing and deploying models. At Next, the company made several additional announcements as well, including updates to its suite of Cloud AI Solutions, AutoML, BigQuery ML, its pre-trained ML APIs, and more. Petuum Unveils Industrial AI Product for Complex Manufacturing Operations. Petuum announced the Petuum Industrial AI Autopilot product, which enables optimization of complex manufacturing operations with modules that continuously learn and adapt. Dollars & Sense Intel Capital announced investments in five AI companies at its annual Global Summit. IntelCapital led a $13M round in AI chip startup Untether AI and a $150M round in AI systems company SambaNova Systems. In addition, the company invested undisclosed amounts in Andrew Ng’s Landing AI and China-based CloudPick and Zhuhai Eeasy Technology. Run.AI, a startup building a new virtualization and acceleration platform for deep learning, announced that it has raised $13M Enlitic, a San Francisco, CA-based company leveraging AI for medical imaging raised $15M in Series B financing Boston Dynamics announced that it is acquiring California-based startup Kinema Systems, which makes computer-vision and ML systems for warehouse robots Evolve IP, announced that it has acquired Jog.ai, a speech analytics and natural language technology firm based in Austin, Texas Rasa, San Francisco-based an open source company that enables developers to build contextual AI assistants, has secured $13M in Series A funding Labelbox, a collaborative training data platform for machine learning applications, has raised a $10M Series A funding Observe.AI has secured $8M in a Series A funding To receive the Bits & Bytes to your inbox, subscribe to our Newsletter.
Bits & Bytes Google announces TensorFlow 2.0 Alpha, TensorFlow Federated, TensorFlow Privacy. At the 3rd annual TensorFlow Developer Summit, Google announced the first alpha release of TensorFlow 2.0 and several other new releases such as: TensorFlow Federated – a new open-source framework that allows developers to use all the ML-training features from TF while keeping the data local; TensorFlow Privacy – which uses differential privacy to process data in a private manner; extensions to TensorFlow Extended (TFX), a platform for end-to-end machine learning; and Activation Atlases – which attempts to visualize and explain how neural networks process images. Google open sources GPipe, a library for parallel training of large-scale neural networks. GPipe, which is based on the Lingvo (a TensorFlow framework for sequence modeling), is applicable to any network consisting of multiple sequential layers and allows researchers to “easily” scale performance. [Paper] Facebook AI researchers create a text-based adventure to study how AI speak and act. Researchers from Facebook and University College London specifically investigated the impact of grounding dialogue – a collection of mutual knowledge, beliefs, and assumptions essential for communication between two people–on AI agents. Google announces Coral platform for building IoT hardware with on-device AI. Coral targets developers creating IoT hardware from prototyping to production. It is powered by a TPU that is specifically designed to run at the edge and is available in beta. Google and DeepMind are using AI to predict the energy output of wind farms. Google announced that it has made energy produced by wind farms more viable using DeepMind’s ML algorithms to better predict the wind output. Ben-Gurion U. develops new AI platform for ALS care. Researchers at Ben-Gurion University have used ML models to develop a new method of monitoring and predicting the progression of neurodegenerative and help identify markers for personalized patient care and improve drug development. Google rolls out AI grammar checker for G Suite users. Google applies ML techniques to understand complex grammar rules and identify “tricky” grammatical errors by G Suite users. Dollars & Sense PolyAI, a London, UK-based platform for conversational AI, raised $12M in Series A funding Wade & Wendy, a NYC-based AI recruitment platform, closed a $7.6M Series A funding Brodmann17, a Tel Aviv, based provider of vision-first technology for automated driving, raised $11M in Series A funding Paradox.ai, a Scottsdale-based assistive intelligence platform raised $13.34M series A funding Apple acquires patents from AI security camera maker Lighthouse Horizon Robotics, China-based AI chip maker raises $600M ELSA, US-based AI language learning app, raised $7M Modulate, a Cambridge-based ML startup raised $2M in seed funding Zone7, which uses AI to predict injuries in sports, has secured $2.5M DataRobot acquires a data collaboration platform company, Cursor Splice Machine announced that it has raised $16M for unified ML platform Senseon has raised $6.4M to tackle cybersecurity threats with an AI ‘triangulation’ approach Ctrl-labs, a New York startup announced that it has raised $28M in a funding round led by GV, Google’s venture capital arm Armorblox, a Sunnyvale, CA-based provider of a natural language understanding platform for cybersecurity, raised $16.5M Series A funding ViSenze, Singapore-based AI startup, has raised $20M in series C funding BlackBerry announces the acquisition of Cylance, a cybersecurity and AI firm To receive the Bits & Bytes to your inbox, subscribe to our Newsletter.
Bits & Bytes IBM takes Watson AI to AWS, Google, Azure. In an effort to broaden the distribution of its AI technology, IBM announced that it is making IBM Watson available across competing public cloud services such as AWS, Google, and Azure. FAIR open-sources new ELF OpenGo data set, research, and insights. Facebook AI Research (FAIR’s) new features and research results include a completely retrained model of ELF OpenGo using reinforcement learning. The open-sourcing of these models provide an important benchmark for future research. Google collaborates with DeepMind to announce PlaNet for reinforcement learning. The Deep Planning Network (PlaNet) agent learns a world model from image inputs and leverages it for planning. It’s able to solve a variety of image-based tasks with up to 5,000 percent the data efficiency. Cogitai launches Continua, a cloud-based, self-learning AI platform for businesses. The SAAS platform for self-learning AI, Continua platform aims to turn any process, system, software bot, or real robot into a self-learning autonomous service. IBM, SUNY Poly creating AI “hardware lab” in Albany. IBM has agreed to extend its presence at SUNY Poly’s Center for Semiconductor Research through 2023, with an option for another five-year agreement after that. IBM’s total spending under the deal will be $2 billion. Google announces Live Transcribe for real-time continuous transcription. Live Transcribe captions conversations in real-time, with support for over 70 languages. Unity announces forthcoming Obstacle Tower Challenge. The new AI challenge aims to test the vision, control, planning, and generalization abilities of intelligent agents in an end-to-end manner. It’s based on Obstacle Tower, a new research environment combining platform-style gameplay with puzzles and planning problems, spanning an endless number of floors that become progressively more difficult as an agent advances. Dollars & Sense Peltarion, a Swedish startup announced that it has raised a $20M Series A funding Polly.ai, a Seattle, based company has raised a $7M Series A funding round TuSimple, a self-driving truck startup has raised $95M in a Series D funding round Apple has acquired AI startup called PullString that specializes in helping companies build conversational voice apps Sensory Inc. announced acquisition of Vocalize.ai, which provides benchmarking, accuracy assessments and bias evaluations for speech technologies and natural language interface Databricks, announced it has raised $250M in a Series E funding round led by Andreessen Horowitz. Coatue Management, Microsoft Aire, a London based credit scoring system company, raised $11M in Series B funding KenSci has raised $22M to fuel the growth of its ML and AI-powered technology New Relic, Inc., provider of real-time insights for software-driven businesses, announced that it has acquired SignifAI Avalara, Inc. announced it has acquired AI technology and expertise from Indix Facebook has acquired visual shopping and AI startup GrokStyle To receive the Bits & Bytes to your inbox, subscribe to our Newsletter.
Sam Charrington: Today we're excited to continue the AI for the benefit of society series that we've partnered with Microsoft to bring to you. Today we're joined by Hanna Wallach principal researcher at Microsoft research. Hanna and I really dig into how bias and a lack of interpretability and transparency show up across machine learning. We discuss the role that human biases, even those that are inadvertent, play in tainting data, whether deployment of fair ML algorithms can actually be achieved in practice and much more. Along the way, Hannah points us to a ton of papers and resources to further explore the topic of fairness in ML. You'll definitely want to check out the show notes page for this episode, which you'll find at twimlai.com/talk/232. Before diving in I'd like to thank Microsoft for their support of the show and their sponsorship of this series. Microsoft is committed to ensuring the responsible development and use of AI and is empowering people around the world with this intelligent technology to help solve previously intractable societal challenges, spanning sustainability, accessibility and humanitarian action. Learn more about their plan at Microsoft.ai. Enjoy. Sam Charrington: [00:02:18] All right everyone, I am on the line with Hanna Wallack, Hanna is a principal researcher at Microsoft Research in New York City. Hanna, welcome to this week in Machine Learning and AI. Hanna Wallach:[00:00:11] Thanks, Sam. It's really awesome to be here. Sam Charrington: [00:00:14] It is a pleasure to have you on the show, and I'm really looking forward to this conversation. You are clearly very well known in the machine learning and AI space. Last year, you were the program chair at one of the largest conferences in the field, NeurIPS. In 2019, you'll be it's general chair. But for those who don't know about your background, tell us a little bit about how you got involved and started in ML and AI. Hanna Wallach:[00:00:48] Sure. Absolutely. So I am a machine learning researcher by training, as you might expect. I've been doing machine learning for about 17 years now. So since way before this stuff was even remotely fashionable, or popular, or cool, or whatever it is nowadays. In that time, we've really seen machine learning change a lot. It's sort of gone from this weirdo academic discipline only of interest to nerds like me, to something that's so mainstream that it's on billboards, it's in TV shows, and so on and so forth. It's been pretty incredible to see that shift over that time. I got into machine learning sort of by accident, I think that's often what happens. I had taken some undergrad classes on information theory and stuff like that, found that to be really interesting, but thought that I was probably going to go into human computer interaction research. But through a research assistantship during the summer between my undergrad degree and my Master's degree, I ended up discovering machine learning, and was completely blown away by it. I realized that this is what I wanted to do. I've been focusing on machine learning in various different forms since them. My PHD was specifically on Bayesian Latent Variable methods, typically for analyzing text and documents. So topic models, that kind of thing. But during my PHD, I really began to realize that I'm not particularly interested in analyzing documents for the sake of analyzing documents, I'm interested in analyzing documents because humans write documents to communicate with one another. It's really that underlying social process that I'm most interested in. So then during my postdoc, I started to shift direction from primarily looking at text and documents to thinking really about those social processes. So not just what are people saying, but also who’s interacting with whom, and thinking about machine learning methods for analyzing the structure and content of social processes in combination. I then dove into this much more when I got a faculty job, because I was hired as part of UMass Amherst’s Computational Social Science Initiative. So at that point I started focusing really in depth on this idea of using machine learning to study society. I established collaborations with a number of different social scientists, focusing on a number of different topics. Over the years, I've mostly ended up working with political scientists, and often study questions relating to government transparency, and still looking at sort of this whole idea of a social process consists of individuals, or groups of individuals interacting with one another, information that might be used in or arising from these interactions, and then the fact that these things might change over time. I often use one of these or two of these modalities, so structure, content, or dynamics, to learn about one or more of the other ones as well. As I continued to work in this space, I started to think more, not just about how we can use machine learning to study society, but the fact that machine learning is becoming much more prevalent within society. About four years ago, I started really thinking more about these issues of fairness, accountability, transparency, and ethics. It was a pretty natural fit for me to start moving in this direction. Not only was I already thinking about questions to do with people, but I've done a lot of diversity and inclusion work in my non research life. So I'm one of the co-founders of the Women in Machine Learning workshop, I also co-founded two organizations to get more women involved in free and open source software development. So issues related to fairness and stuff like that are really something that I tend to think about a lot in general. So I ended up making sort of this shift a little bit in my research focus. That's not to say that I don't still work on things to do with core computational social science, but increasingly my research is focusing on the ways that machine learning impacts society. So fairness, accountability, transparency, and ethics. Sam Charrington: [00:05:53] We will certainly dive deep into those topics. But before we do, you've mentioned a couple of times the term computational social science. That's not a term that I've heard before, I don't believe. Can you ... Is that ... I guess I'm curious how established that is as a field, or is it something that is specific to that institution that you were working at? Hanna Wallach:[00:06:19] Sure. So this is really a discipline that started emerging in maybe sort of 2009, 2008, that kind of time. By 2010, which is when I was hired at UMass, it really was sort of its own little emerging field with a bunch of different computer scientists and social scientists really committed to pushing this forward as a discipline. The basic idea, of course, is you know social scientists study society and social processes, and they've been doing this for decades. But often using qualitative methods. But of course, as more of society moves towards digitized interaction methods, and online platforms, and other kinds of things like that, we're beginning to see much more of this sort of digital data. At the same time, we've seen this massive increase, as I've said, in the popularity of machine learning and machine learning methods that are really suitable for analyzing data about social processes in society. So computational social science is really the sort of emerging discipline at the intersection of computer science, the social sciences, and statistics as well. The real goal is to develop and use computational and statistical methods, so machine learning methods, for example, to understand society, social processes, and answer questions that are substantively interesting to social scientists. At this point, there are people at a number of different institutions focusing on computational social science. So yes, of course, UMass, as I've mentioned before. But also Northwestern, Northeastern, University of Washington, in fact have been doing this for years, and of course, Microsoft Research is no exception in this regard. Part of the reason why I joined Microsoft Research was that we have a truly exceptional group of researchers in computational social science here. That was really very appealing to me. Sam Charrington: [00:08:31] Oh, awesome, awesome. So you talked about your transition to focusing on fairness, accountability, transparency, and ethics in machine learning and AI. Can you talk a little bit about what those terms mean to you, and your broader research? Hanna Wallach:[00:08:54] Yeah, absolutely. So I think the bulk of my own research in that sort of broad umbrella falls within two categories. So the first is fairness, and the second is what I would sort of describe as interpretability of machine learning. So in that fairness bucket, really, much of my research is focused on studying the ways in which machine learning can inadvertently harm or disadvantage groups of people or individual people in various different, usually unintended, ways. I'm interested in understanding not only why this occurs, but what we can do to mitigate it, and what we can do to really develop fairer machine learning systems. So systems that don't inadvertently harm individuals or groups of people. In the intelligibility bucket, so there, I'm really interested in how we can make machine learning methods that are interpretable to humans in different roles for particular purposes. There has been a lot of research in this area over the past few years, focusing on oftentimes developing simple machine learning models that can be easily understood my humans simply by exposing their internals, and also on developing methods that can generate explanations for either entire models or the predictions of models. Those models might be potentially very complex. My own work typically focuses really more on the human side of intelligibility, so what is it that might make a system intelligible or interpretable to a human trying to carry out some particular task? I do a lot of human subjects experiments to really try and understand some of those questions with a variety of different folks here at Microsoft Research. Sam Charrington: [00:11:01] On the topic of fairness and avoiding inadvertent harm, there are a lot of examples that I think many of our audience would be familiar with, the ProPublica work into the use of machine learning systems in the justice process, and others. Are there examples that come to mind for you that are maybe less well known, but that illustrate for you the importance of that type of work? Hanna Wallach:[00:11:36] Yes. So when I typically think about this space, I tend to think about this in terms of the types of different harms that can occur. I have some work with Aaron Shapiro, Solon Barocas, and Kate Crawford on the different types of harms that can occur. Kate Crawford actually did a fantastic job of talking about this work in her invited talk at the NeurIPS conference in 2017. But to give you some concrete examples, so many of the examples that people are most familiar with are these scenarios as you mentioned where machine learning systems are being used to allocate or withhold resources, opportunities, or information. So one example would be of the compass recidivism prediction system being used to make decisions about whether people should be released on bail. Another example would be from a story, a news story that happened in November where Amazon revealed that it had abandoned an automating hiring tool because of fears that the tool would reinforce existing gender imbalances in the workplace. So there you're looking at these existing gender imbalances, and seeing that this tool is perhaps withholding opportunities from women in the tech industry in an undesirable way. There was a lot of coverage about this very sensible decision that Amazon made to abandon that tool. Some other examples would be more related to quality of service issues even when no resources or opportunities are being allocated or withheld. So a great example there would be the work that Joy Buolamwini and Timnit Gebru did focusing on the ways that commercial gender classification systems might perform less well, so less accurate, for certain groups of people. Another example you might think of, let's say, speech recognition systems. You can imagine systems that work really well for people with certain types of accents, or for people with voices at certain pitches. But less well for other people, certainly for me. I'm British, and I have a lisp. I know that oftentimes speech recognition systems don't do a great job of understanding what I'm saying. This is much less of an issue nowadays, but you know, five or so years ago, this was really frustrating for me. Some other examples are things like stereotyping. So here the most famous example of stereotyping in machine learning is Latanya Sweeney's work from 2013, where she showed that advertisements that were being shown on web searches for different people's names would more typically be advertisements that reinforced stereotypes about black criminality when people searched for sort of black sounding names, than when people searched for stereotypically white sounding names. So there the issue is this sort of reinforcement of these negative stereotypes within society by the placement of particular ads for particular different types of searches. So another example of stereotyping in machine learning would be the work done by Joanna Bryson and others at Princeton University on stereotypes in word embeddings. There has also been some similar work done by my colleague, Adam Kalai, here at Microsoft Research. Both of these groups of researchers showed that if you train word embedding methods, so things like Word2Vec, that try and identify a low dimensional embedding for word types based on the surrounding words that are typically used in conjunction with them in sentences, you end up seeing that these word embeddings reinforce existing gender stereotypes. For example, so the word man ends up being embedded much closer to programmer and similarly woman ends up being embedded much closer to homemaker than vice versa. So that would be another kind of example. Then we see other kinds of examples of unfairness and harms within machine learning as well. So for example, over and under representation. So Matthew Kay and some others at the University of Washington have this really nice paper where they show that for professions with an equal or higher percentage of men than women, the image search results are much more heavily skewed towards images of men than reality. So that would be another kind of example. What you'll see from all of these examples that I've mentioned is that they affect a really wide range of systems and types of machine learning applications. The types of harms or unfairness that might occur are also pretty wide ranging as well, going from yes, sure, allocational withholding of resources, opportunities of information, but moving beyond that to stereotyping and representation and so on. Sam Charrington: [00:17:02] So often when thinking about fairness and bias in machine learning and the types of harm that can come about when unfair systems are developed, the kind of all roads lead back to the data itself, and the biases that are inherent in that data. Given that machine learning and AI is so dependent on data, and often much of the data that we have is biased, what can we do about that, and what are the kinds of things that your research is exploring to help us address these issues? Hanna Wallach:[00:17:41] Absolutely. Yeah, so you've hit on a really important point there, which is that in a lot of the sort of public discourse about fairness in machine learning, you have people making comments about algorithms being unfair, or algorithms being biased. Really, I think this misses some of the most fundamental points about why this is such a challenging landscape. So I want to just emphasize a couple of those here in response to your question. So the first thing is that machine learning is all about taking data, finding patterns in that data, and then often training systems to mimic the decisions that are represented within that data. Of course, we know that the society we live in is not fair. It is biased. There are structural disadvantages and discrimination all over the place. So it's pretty inevitable that if you take data from a society like that, and then train machine learning systems to find patterns expressed in that data, and to mimic the decisions made within that society, you will necessarily reproduce those structural disadvantages, that bias, that discrimination, and so on. So you're absolutely right that a lot of this does indeed come from data. But the other point that I want to make is that it's not just from data and it's not from algorithms per se. The issue is really, as I see it, and as my colleagues here at Microsoft Research see it, the issue is really about people and people's decisions at every point in that machine learning life cycle. So I've done some work on this with a number of people here at Microsoft, most recently I put together a tutorial on machine learning and fairness in collaboration with my colleague Jenn Wortman Vaughan. The way we really think about this is that you have to prioritize fairness at every stage of that machine learning lifecycle. You can't think about it as an afterthought. The reason why is that decisions that we make at every stage can fundamentally impact whether or not a system treats people fairly. So I think it's really important when we're thinking about fairness in machine learning to not just sort of make general statements about algorithms being unfair, or systems being unfair, but really to go back to those particular points and think about how unfairness can kind of creep in at any one of those stages. That might be as early as the task definition stage, so when you're sitting down to develop some machine learning system, it's really important to ask the question of who does this take power from, and who does this give power to? The answers to that question often reveal a lot about whether or not that technology should even be built in this first place. Sometimes the answer to addressing fairness in machine learning is simply, no, we should not be building that technology. But there are all kinds of other decisions and assumptions at other points in that machine learning life cycle as well. So the way we typically like to think about it is that a machine learning model, or method, is effectively an abstraction of the world. In making that abstraction, you necessarily have to make a bunch of assumptions about the world. Some of these assumptions will be more or less justified, some of these assumptions will be better fit for the reality than others. But if you're not thinking really carefully about what those assumptions are, when you are developing your machine learning system, this is one of the most obvious places that you can inadvertently end up introducing bias or unfairness. Sam Charrington: [00:21:42] Can you give us some concrete examples there? Hanna Wallach:[00:21:45] Yeah. Absolutely. One common example of this form would be stuff to do with teacher evaluation. So there have been a couple of high profile lawsuits about this kind of thing. But I think it illustrates the point nicely. So it's common for teachers to be evaluated based on a number of different factors, but including their student's test scores. Indeed, many of the methods that have been developed to analyze teacher quality using machine learning systems have really focused predominantly on student's test scores. But this assumes that student's test scores are in fact an accurate predictor of teacher quality. This isn't actually always the case. A good teacher should obviously do more than test prep. So any system that really looks just at test scores when trying to predict teacher quality is going to do a bad job of capturing these other properties. So that would be one example. Another example involves predictive policing. So a predictive policing system might make predictions about where crimes will be committed based on historic arrest data. But an implicit assumption here is that the number of arrests in an area is an accurate proxy for the amount of crime. It doesn't take into account the fact that policing practices can be racially biased, or there might be historic over policing in less affluent neighborhoods. I'll give you another example as well. So many machine learning methods work by defining some objective function, and then learning the parameters of the model so as to optimize that objective function. So for example, if you define an objective function in the context of, let's say, a search engine, that prioritizes user clicks, you may end up with search results that don't necessarily reflect what you want them to. This is because users may click on certain types of search results over other search results, and that might not be reflective of what you want to be showing when you show users a page of search results. So as a concrete example, many search engines, if you search for the word boy, you see a bunch of pictures of male children. But if you search for the world girl, you see a bunch of pictures of grown up women. These are pretty different to each other. This probably comes from the fact that search engines typically optimize for clicks among other metrics. This really shows how hard it can be to even address these kinds of fairness issues, because in different circumstances the word girl may be referring to a child or a woman, and users search for this term with different intentions. In this particular example, as you can probably imagine, one of these intentions might be more prevalent than the other. Sam Charrington: [00:24:57] You've identified lots of opportunities for pitfalls in the process of fielding systems going all the way back to the way you define your system, and state your intentions, and formulate the problem that you're going after. Beyond simply being mindful of the potential for bias and unfairness and just saying simply, I realize that that's not simple, that it's work to be mindful of this. But beyond that, what does your research offer in terms of how to overcome these kinds of issues? Hanna Wallach:[00:25:43] Yeah, this is a really good question. It's a question that I get a lot from people is what can we actually do in practice? There are a number of things that can be done in practice. Not all of them are easy things to do, as you say. So one of the most important things is that issues relating to fairness in machine learning are fundamentally socio-technical. They're not going to be addressed by computer scientists or developers alone. It's really important to involve a range of diverse stakeholders in these conversations when we're developing machine learning systems so that we have a bunch of different perspectives represented. So moving beyond just involving computer scientists and developers on teams, it's really important that we involve social scientists, lawyers, policy makers, end users, people who are going to be affected or impacted by these systems down the line, and so on and so forth. That's one really concrete thing you can do. There is a project that came out of the University of Washington called the Diverse Voices project. It provides a way of getting feedback from stakeholders on tech policy documents. It's really good, they have a great how-to guide that I definitely recommend checking out. But many of the things that they recommend doing there, you can also think about when you're trying to get feedback from stakeholders on, let's say, the definition of a machine learning system. So that task definition stage. Some of these could even potentially be expanded to consider other stages of that machine learning pipeline as well. So there are a number of things that you can do at every single stage of the machine learning pipeline. In fact, this tutorial that I mentioned earlier, that I worked on with my colleague Jenn Wortman Vaughan actually has guidelines for every single step of the pipeline. But to give you examples, here are some things, for instance, that you can do when you're selecting a data source. So for example, it's really important to think critically before even collecting any data. It's often very tempting to say, oh, there is already some dataset that I can probably repurpose for this. But it's really important to take that step back and before immediately acting based on availability to actually think about whether that data source is appropriate for the task you want to use it for. There is a number of reasons why it might not be, it could be to do with biases and the data source selection process. There might be societal biases present in the data source itself. It might be that the data source doesn't match the deployment context, that's a really important one that people really should be taking into account. Where are you thinking about deploying your machine learning system and does the data you have availability for training and development match that context? As another example, still related to data, it's really important to think about biases in the technology used to collect data. So as an example here, there was an app released in the city of Boston back in 2011, I think it was called Street Bump. The way it worked is it used iPhone data and specifically the sort of positional movement of iPhones as people were driving around, to gather data on where there were potholes that should be repaired by the city. But pretty quickly, the city of Boston figured out that this actually wasn't a great way to get that kind of data, because back in 2011, the people who had iPhones were typically quite affluent and only lived in certain neighborhoods. So that would be an example about thinking carefully about the technology even used to collect data. It's also really important to make sure that there is sufficient representation of different subpopulations who might be ultimately using or affected by your machine learning system to make sure that you really do have good representation overall. Moving onto things like the model, there is a number of different things that you can do there, for instance, as well. So in the case of a model, I mentioned a bit about assumptions being really important. It's great to really clearly define all of your assumptions about the model, and then to question whether there might be any explicit or implicit biases present in those assumptions. That's a really important thing to do when you're thinking about choosing any particular model or model structure. You could even, in some scenarios, include some quantitative notion of parity, for instance, in your model objective function as well. There have been a number of academic papers that take that approach in the literature over the past few years. Sam Charrington: [00:30:43] Can you give an example of that last point? Hanna Wallach:[00:30:46] Yeah, sure. So imagine you have some kind of a machine learning classifier that's going to make decisions of the form, let's say loan, no loan, hire, no hire, bail, no bail, and so on. The way we normally develop these classifiers is to take a bunch of labeled data, so data points labeled with, let's say, loan, no loan, and then we train a model, a machine learning model, a classifier, to optimize accuracy on that training data. So you end up setting the parameters of that model such that it does a good job of accurately predicting those labels from the training data. So the objective function that's typically used is one that considers, usually, only accuracy. But something else you can do is define some quantitative definition of fairness, some quantitative fairness metric, and then try to simultaneously optimize both of these objectives. So classifier accuracy and whatever your chosen fairness metric is. There is a number of these different quantitative metrics that have been proposed out there that all typically are looking at parity across groups of some sort. So I think it's really important to remember that even though these are often referred to as fairness metrics, they're really parity metrics. They neglect many of the really important other aspects of fairness, like justice, and due process, and so on and so forth. But, it is absolutely possible to take these parity metrics and to incorporate them into the objective function of, say, a classifier, and then to try and prioritize satisfying and optimizing that fairness metric at the same time as optimizing classifier accuracy. There have been a number of papers that focus on this kind of approach, many of them will focus on one particular type of classifier, so like SBMs, or neural networks, or something like that, and one particular fairness metric. There are a bunch of standard fairness metrics that people like to look at. I actually have some work with some colleagues here at Microsoft where we have a slightly more general way of doing this that will work with many different types of classifiers, and many different types of fairness metrics. So there is no reason to start again from scratch if you want to switch to a different classifier or a different fairness metric. We actually have some open source Python code available on GitHub that implements our approach. Sam Charrington: [00:33:27] So you've talked about the idea that kind of people are fundamentally the root of the issue, that these are societal issues, that they're not going to be solved by technological advancements or processes alone. At the same time, there has been a ton of new research happening in this area by folks in your group and elsewhere. Does that lead to a mismatch between what's happening in academia and on the technical side with the way this stuff actually gets put into practice? Hanna Wallach:[00:34:11] That's an awesome question. The simple answer is yes. This actually relates to one of my most recent research projects, which I'm really, really excited about. So last summer, some of my colleagues and I, specifically Jenn Wortman Vaughan, Miro Dudík, and Hal Daumé, along with our incredible intern, Ken Holstein from CMU, conducted the first systematic investigation of industry practitioner's challenges and needs for support relating to developing fairer machine learning systems. This work actually came about because we were thinking about ways of developing interfaces for that fair classification work that I mentioned a minute ago. Through a number of conversations with people in different product groups here at Microsoft and people at other companies, we realized that these kinds of classification tasks, while they're incredibly well studied within the fairness and machine learning literature, are maybe less common than we had thought in practice within industry. So that got us thinking about whether there might be, actually, a mismatch between the academic literature on fairness and machine learning, and practitioner's actual needs. What we ended up doing was this super interesting research project that was a pretty different style of research for me and for my colleagues. So I am a machine learning researcher, so is Jen, so is Howell, and so is Miro. Ken, our intern, is an HCI researcher. What we ended up doing was this qualitative HCI work to really understand what it is that practitioners are facing in reality when they try and develop fairer machine learning systems. To do this, we conducted semi structured interviews with 35 people, spanning 25 different teams, in 10 different companies. These people were in a number of different roles, ranging from social scientist, data labeler, product manager, program manager, to data scientists and researcher. Where possible, we tried to interview multiple people from the same team in order to get a variety of perspectives on that team's challenges and needs for support. We then took our findings from these interviews, and developed a survey which was then completed by another 267 industry practitioners, again, in a variety of different companies and a variety of different roles. What we found, at a high level, was that yes, there is a mismatch between the academic literature on fairness in machine learning and industry practitioner's actual challenges and needs for support on the ground. So firstly, much of the machine learning literature on fairness focuses on classification, and on supervised machine learning methods. In fact, what we found is that industry practitioners are grappling with fairness issues in a much wider range of applications beyond classification or prediction scenarios. In fact, many times the systems they're dealing with involve these really rich, complex interactions between users and the system. So for example, chat bots, or adaptive tutoring, or personalized retail, and so on and so forth. So as a result, they often struggle to use existing fairness research from the literature, because the things that they're facing are much less amenable to these quantitative fairness metrics. Indeed, very few teams have fairness KPIs or automated tests that they can use within their domain. One of the other things that we found is that the machine learning literature typically assumes access to sensitive attributes like race or gender, for the purpose of auditing systems for fairness. But in practice, many teams have no access to these kinds of attributes, and certainly not at the level of individuals. So they express needs for support in detecting biases and unfairness with access only to core screened, partial, or indirect information. This is something that we've seen much less focus on in the academic literature. Sam Charrington: [00:38:41] That last point is an interesting one, and one that I've brought up on the podcast previously. In many of the places you might want to use an approach like that, it's forbidden, from a regulatory perspective, to use the information that you want to use in your classifier to achieve fairness in any part of the decisioning process. Hanna Wallach:[00:39:04] Exactly. This sets up this really difficult tension between doing the right thing in practice from a machine learning perspective, and what is legally allowed. I'm actually working on a paper at the moment with a lawyer, Zack Conard, actually, a law student, Zack Conard, at Stanford University, on exactly this issue. This challenge between what you want to do from a machine learning perspective, and what you are required to do from a legal perspective, based on humans and how humans behave, and hundreds of years of law in that realm. It's really challenging, and there is this complicated trade off there that we really need to be thinking about. Sam Charrington: [00:39:48] It does make me wonder if techniques like or analogous to a differential privacy or something like that could be used to provide a regulatorily acceptable way to access protected attributes, so that they can be incorporated into algorithms like this. Hanna Wallach:[00:40:07] Yeah, so there was some work on exactly this kind of topic at the FAT ML Workshop colocated with ICML last year. This work was proposing the use of encryption and such like in order to collect and make available such information, but in a way that users would feel as if their privacy was being respected, and so that people who wanted to use that information would be able to use it for purposes such as auditing. I think that's a really promising approach, although there is obviously a bunch of non trivial challenges involved in thinking about how you might make that a reality. It's a really complicated landscape. But definitely one that's worth thinking about. Sam Charrington: [00:40:54] Was there a third area that you were about to mention? Hanna Wallach:[00:40:58] Yeah, so one of the main themes that we found in our work studying industry practitioners is a real mismatch between the focus on different points in the machine learning life cycle. So the machine learning literature typically assumes no agency over data collection. This makes sense, right? If you're a machine learning academic, you typically work with standard data sets that have been collected and made available for years. You don't typically think about having agency over that data collection process. But of course, in industry, that's exactly where practitioners often do have the most control. They are in charge of that data collection or data curation process, and in contrast, they often have much less control over the methods or models themselves, which often are embedded within much bigger systems. So it's much harder to intervene from a perspective of fairness with the models than it is with the data. We found that really interesting, this sort of difference in emphasis between models versus data in these different groups of people. Of course, many practitioners voiced needs for support in figuring out how to leverage that sort of agency over data collection to create fairer data sets for use in developing their systems. Sam Charrington: [00:42:20] So you mentioned the FAT ML workshop. I'm wondering as we come to a close, if there are any resources, events, pointers, I'm sure there are tons of things that you'd love to point people at. But what are your top three or four things that you would suggest people take a look at as they're trying to wrap their heads around this area, and how to either have an impact as a researcher, or how to make good use of it as a practitioner? Hanna Wallach:[00:42:55] Yeah. Absolutely. So there are a number of different places with resources to learn more about this kind of stuff. So first, I've mentioned a couple of times, this tutorial that I put together with Jen Waltman Vahn, that will be available publicly online very soon. It is in fact being broadcast next week, so it should be up by the time this podcast goes live. So I would definitely recommend that people check that out to really get a sense of how we, at Microsoft, are thinking about fairness in machine learning. Then moving beyond that, and thinking specifically on more of the academic literature, the FAT ML workshop maintains a list of resources on the workshop website. That's again, another really, really great place to look for things to read about this topic. The FAT Star conference is a relatively newly created conference on fairness accountability and transparency, not just in machine learning, but across all of computer science and computational systems. Again, there, I recommend checking out the website to see the publications that were there last year, and also the publications that will be there this year. There is a number of really interesting papers that I haven't read yet, but I'm super excited to read, being presented at this year's conference. That conference also has tutorials on a range of different subjects. So it's also worth looking at the various different tutorials there. So at last year's conference, Arvind Narayanan presented this amazing tutorial on quantitative fairness metrics, and why they're not a one size fits all solution, why there are trade offs between them, why you can't just sort of take one of these definitions, optimize for it, and call it quits. So I definitely recommend checking that out. Some other places that are worth looking for resources on this, the AI Now Institute, which was co-founded by Kate Crawford, who is also here at Microsoft Research, and Meredith Whitaker, who is also at Google, also has some incredibly awesome resources. They've put out a number of white papers and reports over the past couple of years that really get at the crux of why these are complicated socio-technical issues. So I strongly recommend reading pretty much everything that they put out. I would also recommend checking out some of the material put out by Data and Society, which is also an organization here in New York, led by Danah Boyd, and they too have a number of really interesting things that you can read about these different topics. Then the final thing I want to emphasize is the Partnership on AI, which was formed a couple of years ago by Microsoft and a bunch of other companies working in this space of AI to really foster cross company collaboration and moving forward in this space when thinking about these complicated societal issues that relate to AI and machine learning. So the partnership has been really ramping up over the past couple of years, and they also have some good resources that are worth checking out. Sam Charrington: [00:46:22] Oh, that's great. That is a great list that will keep us busy for a while. Hanna, thank you so much for taking the time to chat with us. It was really a great conversation, and I appreciate it. Hanna Wallach:[00:46:34] No problem. Thank you for having me. This has been really great. Sam Charrington: [00:46:38] Awesome, thank you.
Bits & Bytes Facebook and Google researchers build RL framework to study language evolution. Their research paper proposes a computational framework allowing agents to interact in a series of games and uses it to demonstrate that symmetric communication protocols (i.e. languages) emerge and evolve without any innate, explicit mechanisms built in the agent. Google releases synthetic speech dataset to help researchers combat “deep fakes.” Google has published a dataset of synthetic speech containing thousands of phrases spoken by deep learning based text-to-speech models. By training models on both real and computer-generated speech, researchers can develop systems that learn to distinguish between the two. Carbon Relay to optimize energy efficiency in data centers with AI. Carbon Relay launched a new data center energy management product using deep reinforcement learning and augmented intelligence to offer customers energy efficiency improvements. Google announced its success with a similar internal project last year. AWS open sources Neo-AI project to accelerate ML on edge devices. Recall that the project extends TensorFlow, MXNet, PyTorch, ONNX, and XGBoost models to perform at up to twice the speed of the original model with no loss in accuracy on multiple hardware platforms. It’s based on the TVM and Treelite compilers developed at the University of Washington. Microsoft and MIT work to detect ‘blind spots’ in self-driving cars. A model developed by MIT and Microsoft researchers identifies instances where autonomous cars have learned actions from training examples that could cause real-world errors on the road. Amazon facial-identification software used by police falls short on tests for accuracy and bias. AWS may be digging itself a deeper and deeper hole as it attempts to refute claims of bias for its facial-recognition software, Rekognition, marketed to local and federal law enforcement as a crime-fighting tool, struggles to pass basic tests of accuracy. Spell expands cloud AI platform. The Spell platform uses Kubernetes to automatically scale models as necessary and provides metrics, monitoring, and logs for everything running in real time. Its latest edition adds team and collaboration features. The company also announced new funding; see below. Dollars & Sense Israel-based start-up, Augury which provides AI solution to predict industrial equipment failure, raised $25M in a Series C funding Aureus Analytics, a predictive analytics platform for the insurance industry, has secured $3.1M in funding Mimiro, a London, UK-based ML platform for analyzing the risk of financial crime, raised $30M in Series B funding Carbon Relay, a Boston and Washington, D.C.based company raised $5M to tackle data center cooling with AI Verbit Software Ltd, a provider of automated video and speech transcription services powered by AI has raised $23M round of funding led by Viola Ventures Cinnamon AI, a provider of AI solutions, has secured $15M in Series B funding Sonasoft Corp. announces that it has signed a letter of intent to acquire Hotify Rover180 has acquired an AI automation and ML Indiana-based company, Vemity Sherpa.ai, Palo Alto, California-based AI-powered digital predictive assistant provider, raised $8.5M in Series A funding AInnovation, a Chinese AI solutions provider raised approximately $60M in Series A and A+ financing round Zeta Global has acquired Silicon Valley-based AI company, Temnos Adjust announced that it has entered into a definitive agreement to acquire cybersecurity and AI start-up, Unbotify Spell closed $15M in new funding from Eclipse Ventures and Two Sigma Ventures. To receive the Bits & Bytes to your inbox, subscribe to our Newsletter.
Bits & Bytes Google introduces Feast: an open source feature store for ML. GO-JEK and Google announced the release of Feast that allows teams to manage, store, and discover features to use for ML projects. Amazon CEO, Jeff Bezos, is launching a new conference dedicated to AI. The new AI specific conference, re:MARS, will be held in Las Vegas between June 4th and 7th this year. Should be an interesting event. Mayo Clinic research uses AI for early detection of silent heart disease. Mayo Clinic study finds that applying AI to electrocardiogram (EKG) test results offers a simple, affordable early indicator of asymptomatic left ventricular dysfunction, a precursor to heart failure. Microsoft announces ML.NET 0.9. Microsoft’s open-source and cross-platform ML framework, ML.NET, was updated to version 0.9. New and updated features focus on expanded model interpretability capabilities, GPU support for ONNX models, new Visual Studio project templates in preview, and more. Intel and Alibaba team up on new AI-powered 3D athlete tracking technology. At CES 2019, Intel and Alibaba announced the new collaboration to develop AI-powered 3D athlete tracking technology to be deployed at the 2020 Olympic Games. Baidu unveils open source edge computing platform and AI boards. OpenEdge, an open source computing platform enables developers to build edge applications with more flexibility. The company also announced new AI hardware development platforms BIE-AI-Box with Intel for in-car video analysis, and BIE-AI-Board, co-developed with NXP, for object classification. Qualcomm shows off an AI-equipped car cockpit at CES 2019. At CES, Qualcomm introduced the third generation of its Snapdragon Automotive Cockpit Platforms. The upgraded version covers various aspects of the in-car experience from voice-activated interfaces to traditional navigation systems. Their keynote featured a nice demo of “pedestrian intent prediction” based on various computer vision techniques including object detection and pose estimation. Dollars & Sense Fractal Analytics, an AI firm based in India which, among other things, owns Qure.ai (see my interview with CEO Prashant Warier), raised $200M from private equity investor Apax Standard has acquired Explorer.ai, a mapping and computer vision start-up Israeli AI-based object recognition company, AnyVision, has raised $15M from Lightspeed Venture Partners Spell, an NYC-based AI and ML platform startup raised $15M HyperScience, an edge ML company has raised $30M WeRide.ai, a Chinese autonomous driving technology specialist raised series A funding from SenseTime Technology and ABC International in series A funding UK-based Exscientia, AI-driven drug discovery company, has raised $26 million CrowdAnalytix raises $40 million in strategic investment for crowdsourced AI algorithms To receive the Bits & Bytes to your inbox, subscribe to our Newsletter.
Bits & Bytes Microsoft leads the AI patent race. As per EconSight research findings, Microsoft leads the AI patent race going into 2019 with 697 patents that the firm classifies as having a significant competitive impact as of November 2018. Out of the top 30 companies and research institutions as defined by EconSight in their recent analysis, Microsoft has created 20% of all patents in the global group of patent-producing companies and institutions. AI hides data from its creators to cheat at its appointed task. Research from Stanford and Google found that the ML agent intended to transform aerial images into street maps and back was found to be hiding information it would need later. Tech Mahindra launches GAiA for enterprises. GAiA is the first commercial version of the open source Acumos platform, explored in detail in my conversation with project sponsor Mazin Gilbert about a year ago. Taiwan AI Labs and Microsoft launch AI platform to facilitate genetic analysis. The new AI platform “TaiGenomics” utilizes AI techniques to process, analyze, and draw inferences from vast amounts of medical and genetic data provided by patients and hospitals. Google to open AI lab in Princeton. The AI lab will comprise a mix of faculty members and students. Elad Hazan and Yoram Singer, who both work at Google and Princeton and are co-developers of the AdaGrad algorithm, will lead the lab. The focus of the group is developing efficient methods for faster training. IBM designs AI-enabled fingernail sensor to track diseases. This tiny, wearable fingernail sensor can track disease progression and share details on medication effectiveness for Parkinson’s disease and cardiovascular health. ZestFinance and Microsoft collaborate on AI solution for credit underwriting. Financial institutions will be able to use the Zest Automated Machine Learning (ZAML) tools to build, deploy, and monitor credit models using the Microsoft Azure cloud and ML Server. Dollars & Sense Swiss startup  Sophia Genetics raises $77M to expand its AI diagnostic platform Baraja, LiDAR start-up, has raised $32M in a series A round of funding Semiconductor firm QuickLogic announced that it has acquired SensiML, a specialist in ML for IoT applications Donnelley Financial Solutions announced the acquisition of eBrevia, a provider of AI-based data extraction and contract analytics software solutions Graphcore, a UK-based AI chipmaker, has secured $200M in funding, investors include BMW Ventures and Microsoft Dataiku Inc, offering an enterprise data science and ML platform, has raised $101M in Series C funding Ada, a Toronto-based co focused on automating customer service, has raised $19M in funding To receive the Bits & Bytes to your inbox, subscribe to our Newsletter.
Happy New Year! I've spent the week at CES in Las Vegas this week, checking out a bunch of exciting new technology. (And a bunch of not-so-exciting technology as well.) I'll be writing a bit more about my experiences at CES on the TWIML blog, but for now I'll simply state the obvious: AI was front and center at this year's show, with many interesting applications, spanning smart home and city to autonomous vehicles (using the term vehicle very broadly) to health tech and fitness tech. I focused on making videos this time around, and we'll be adding a bunch from the show to our CES 2019 playlist over on Youtube, so be sure to check that out and subscribe to our channel while you're there. In other news, we just wrapped up our AI Rewind 2018 series in which I discussed key trends from 2018 and predictions for 2019 with some of your favorite TWIML guests. This series was a bit of an experiment for us and we're excited to have received a lot of great feedback on it. If you've had a chance to check it out I'd love to hear your thoughts. Cheers, Sam P.S. We're always looking for sponsors to support our work with the podcast. If you think your company might benefit from TWIML sponsorship, I'd love your help getting connected to the right people. P.P.S. I'm planning a visit to the Bay Area for the week of January 21st. I've got a few open slots for briefings and meeting up with listeners. If you're interested in connecting give me a holler.   To receive this to your inbox, subscribe to our Newsletter.