Explainable AI for Transparency and Trust

Teams frequently face “build or buy” decisions when evaluating the cost-to-benefit ratio of using external vendors versus investing in building something in-house. In our previous experience, one frequent consideration that would come up is whether the onboarding and maintenance of a new service would cost more time and effort than not having integrated it at all.

This was a particularly sensitive matter when it came to software services that generated recommendations for fraud or abuse detection using analytics or predictive AI. As expected, these systems sometimes produced incorrect results. While it is expected that no system is perfect, one consistent source of frustration encountered by our teams was the lack of transparency around how a given recommendation was made by a selected tool, costing engineers, data scientists, and analysts countless hours spent investigating black box decisions.

For this reason, we chose to make Nuanced a product that not only yields an overall evaluation of whether a given image is likely generated by AI, but also provide some level of interpretability for said decision.

What is explainability, and why does it matter?

In the context of AI, explainability refers to the “how” and “why” behind automated decisions generated by models. This may include information regarding which features a model weighs more heavily than others, as well as assumptions and logic that guide its output. This not only provides insight into factors that influence results, but can also facilitate human-in-the-loop decision processes, allowing human evaluators to interpret results and gain improved understanding. This is especially important when the output informs actions with high-stakes ramifications, such as when to suspend a user.

Explainability can also help spot anomalies and patterns over time, further enhancing decision makers’ ability to leverage models.

Nuanced Explainability

No matter how much we work to enhance our accuracy and precision, it would be naive to assume we will always get the right answer. For this reason, Nuanced provides insight into how we came to the decision in two ways: (1) how confident we are, and (2) which parts of the image are more likely to be AI-generated versus “real”.

1. Confidence. Rather than providing a binary decision as to whether something is AI-generated or not, we ensure that an overall evaluation is accompanied by a confidence score. An output suggesting an image is 51% likely to be AI-generated is very different compared to one suggesting an image is 99% likely to be AI-generated. The sensitivity to which models are tuned to reflect a confidence in line with what we, as humans, understand as confidence, since neural networks are generally over-confident in their output scores.
2. Saliency. We also provide information about which parts of an image contributed to the model’s decision. This is because some images are entirely AI-generated, while others have edit sub-regions. Making the distinction between the two, and having a way to identify parts that are likely to be AI-generated is useful in contexts where, for example, a listing for a property rental may use a real landscape and edit in a vacation home that doesn’t exist in real life. Consider the image below demonstrating (1) an original, real image of my cat, (2) an AI-generated cowboy hat, and (3) the overlay showing the parts that are likely to be AI-generated.

We also believe that explainability is a UX problem. Understanding what a “good” explanation is and providing the appropriate information or selecting the visual presentation around how we came to a decision.

Transparency, trust, and usability

When purchasing any off-the-shelf solution, it isn’t enough that it automates a task, but that the automation yields useful results and increases productivity, rather than hampering it.

Explainability can build trust in a variety of ways. Greater transparency around AI models can enhance our collective understanding of their reasoning, as well as their biases and limitations. This can ensure deployment of such systems is ethical and responsible.