Machine learning has seen many dramatic successes, but there's often a missing ingredient. Some of the most successful models are also some of the hardest to understand. Not only does this problem have obvious intellectual interest, it has many real-world ramifications. Transparency is critical for interpreting, debugging, and controlling machine learning systems.

As part of the Google Brain team, my colleagues and I have worked to create tools for inspecting the workings of ML models. We have also seen that training data is often a key to understanding—a point of view summarized in the slogan, "Don't just debug the model, debug the data."

The image above shows the Embedding Projector, a visualization tool for rich interactive exploration of the kind of high-dimensional data sets that are common in machine learning. My colleagues and I have used this tool as a kind of scientific instrument, leading to insights into state-of-the-art systems. For examples, an investigation of a machine translation model found suggestions of a language-independent representation of meaning.

A second challenge is to explore the actions of complex ML models in the real world. The power—and challenge—of ML systems is that their behavior is not predefined by a human. The image below shows an application we created for monitoring changes in a large-scale mission critical ML system. Here we attacked the problem of understanding how a difference in model corresponds to a change in output.



A final challenge is to help engineers and scientists learn about ML systems themselves. One example of my team's work in this area is the TensorFlow Playground, which allows users to play with a neural network in their browser, no programming required.