Understanding the Insights Supply Chain

The Insights Supply Chain describes how to lead data professionals and organize their teams to generate insights from data. This framework establishes organizing principles for empowering data professionals across a range of expertise and data domains. Just as the people of the Insights Supply Chain have profiles and personalities, the data that flow through the Insights Supply Chain have properties. Data professionals typically shape these properties based on their position in the organization. For example, decentralized teams of specialists can focus on crafting data for needs specific to their teams and domain. Centralized teams can focus on standardized data and data flows to facilitate efficient distribution and use across multiple business units.

According to Sarah Callaghan, editor of ‘Patterns’, a cross-disciplinary journal published by Cell Press, these data can be categorized as artisanal and industrial. Understanding this distinction is key to seeing how creativity and structure can coexist in the analytics work of the Insights Supply Chain.

From Data Types to Team Types

Callaghan provides functional descriptions in “Artisanal and Industrial: The Different Methods of Data Creation.” Artisanal data are “the output from single researchers or small teams, working in (relative) isolation” where the data are “generally organized in the way that suits the creator best, using the most convenient and effective tools and software for them.” In the “Data Org Matrix (DOM)” of the Insights Supply Chain, artisanal data will be most prevalent in decentralized teams of specialists. Callaghan describes industrial data as “generally standardized, within the domain of production at least….and generally has data managers whose full-time jobs are to think about and care for the data.” While anyone can use industrial data, it is most relevant and critical within centralized teams who cannot effectively collaborate and deliver using non-standardized data infrastructure and flows. The chart below overlays these data types on the DOM…

Data creation methods align with organizational design: artisanal data thrive in autonomous structures; industrial data sustain organizational structures that rely on consistency and scale.

The Artisan and the Factory: Organizing Analytics Work

While Callaghan classifies the data, analytics expert and leader John Thompson profiles the people who produce insights from the data. Thompson uses the artisanal vs industrial data concept when describing two types of models for organizing data science teams: the artisan model and the factory model. Based on his extensive experience, Thompson uses these models to describe two approaches for building “advanced analytics teams.” The explanatory graphic below comes from one of his many talks on this subject:

Artisan data scientists are highly skilled experts capable of working relatively independently in small teams. They cover almost the entire length of the Insights Supply Chain. These teams take on business problems while using the languages of data engineering, data analytics, and insights analytics which itself directly integrates with the language of business.

The artisan model is commonly equated interchangeably with the data scientist as seen in studies such as “Data Scientists as Craft Workers: Theorizing Data Work” (Konstantin Hopf, Mayur P. Joshi, Arisa Shollo, Marta Stelmaszak). Based on their survey of 62 globally distributed data scientists, these researchers characterize data scientists as data workers who “not only craft the products (analytical models), but also the tools (algorithms), and the material (data)…they also need domain knowledge to understand the complex social phenomena and creative skills to generate novel and relevant insights through the models they build.” With such a range and depth of expertise these craft workers align with Thompson’s operational concept of the artisan data scientist.

Thompson’s factory model describes a sequential or modularized work flow where data professionals in differentiated functions transform the data into insights through a series of hand-offs. The Insights Supply Chain provides a flexible data strategy for organizing such a classic division of labor based on the degree of specialization shared across data professionals. For example, a factory-like production will likely occur in a centralized data team. If the data professionals are generalists, then the data factories can launch and decommission according to a given project. Specialist data professionals may stand up standardized data factories as a source of standardized data products and services as in the specialized/decentralized quadrant in the DOM below.

When Analytics Scales: From Artisanal to Autonomous

The concept of an artisan also appears in the work of analytics scholar Thomas Davenport, who envisions the evolution of analytics from the artisanal to the autonomous. In “Move Your Analytics Operation from Artisanal to Autonomous” (Harvard Business Review, 2016), Davenport introduces the idea of a “Model Factory” that can analyze data at a scale beyond the limits of artisanal methods. Machine learning, he notes, shifts focus from designing models to assembling data and monitoring results for “relevance and reasonability.” Where Davenport’s Model Factory automates, the Insights Supply Chain coordinates in a design where autonomy and standardization coexist as organizational options.

Davenport’s framing resembles Thompson’s. Both analytics experts draw on the artisanal-industrial metaphor, but they diverge in implication. For Davenport, artisanal data work is limiting in a world that demands scale. For Thompson, the artisan achieves scale through mastery and appropriate tools. Where Davenport advocates a transition to factory-like automation, Thompson claims the artisan can harness the same technologies without losing craftwork.

In fact, Thompson’s hybrid model combines the craft of the artisan data scientist and the scalability of factory-type work. In the hybrid model artisans coordinate and assign various (presumably modularized) tasks to the data professionals working in a factory model. Thus an artisan could work with a decentralized business unit that finds itself data-deficient for a given project. This artisan would assemble the team according to their knowledge of the roles distributed across the organization as in the generalized/decentralized quadrant of the DOM.

Redefining the Data Artisan

In 2012, analytics platform company Alteryx apparently first coined the term “data artisan” as part of staking claim to a strategic analytics practice. In “The Definitive Guide to Strategic Analytics“, Alteryx refashions the entire practice of data analytics to the status of artisan…

“Today’s leading-edge data analysts are more artist than reporter, applying creativity and insight into their role in strategic analytics. We like to call them ‘data artisans’. Just as other artisans, data artisans apply skill to their craft, embedding their deeply embedded knowledge of strategic analytics into the analytic process and associated applications. They not only understand their organization’s business drivers and problems, but also where to find the right data for every strategic decision.”

A related 2013 Fast Company article muddied this definition by establishing an unfortunate distinction between the data artisan and the data scientist: “Data artisans are employees who possess a blend of technical skills and business acumen that enables them to extract actionable insight from the huge volumes of data that exist–despite their lack of experience with it–demonstrating that businesses don’t always need a data scientist to interpret data effectively.” In other words, the data artisan is actually someone who is not a data professional by trade but is able to use enough of the tools of the trade to deliver insights without the direct help of other data professionals. In the Insights Supply Chain this person is most likely an Insights Analyst (who DOES call on other data professionals to fill in any technical gaps).

Alteryx’s data artisan appeals to an artistic side of data work. Thompson’s data artisan is a data scientist. The Insights Supply Chain positions these roles much differently. Alteryx’s data artisan lives downstream at the end of the chain. Thompson’s artisan may span the entire chain. I favor Thompson’s model of the artisan because it brings additional richness to the Insights Supply Chain. I previously only considered generalists to be capable of providing coverage across the Insights Supply Chain.

These various definitions illustrate how the term ‘artisan’ can evolve from a marketing metaphor into a tool for describing data leadership and team design in the Insights Supply Chain.

How the Artisan Operates Across the Insights Supply Chain

As shown below, my model of spanning the Insights Supply Chain uses generalization where the roles of data professionals are distributed across teams or the entire organization. Since an artisan data scientist is a specialist with generalizable skills, the artisan can fill each of the roles.

As an example, I once led a centralized analytics team of specialists working in parallel with a team of data scientists with their own leader. Together, we functioned much like Thompson’s artisans, particularly since the company was too small to support a factory model of insight production. Over time, the data scientists evolved into archetypal artisans by expanding their responsibilities upstream to include much of their own data engineering. While spanning the entire Insights Supply Chain, they maintained a sharp focus on model-building, deep analysis, and innovation.

An organization fortunate enough to have artisan data scientists has staffing optionality. These artisans can also help alleviate the critical impacts of generalization. They can command the effort needed to prevent elongated innovation cycles, redundancy, and conflict in defining metrics. The artisan data scientist only needs larger teams in the hybrid model. Data and analytics leaders can use this framework to identify which roles require artisanal breadth and which benefit from factory-like specialization.

Conclusion: The Artisan Advantage in the Insights Supply Chain

At this point, one might conclude that the metaphors relying on production systems are ill-defined, sometimes well-confined, other times expansive. However, as a production metaphor, the Insights Supply Chain provides a complete organizational framework for transforming data into insights. I purposely designed it for flexibility and to answer strategic questions of centralizing versus decentralizing teams of data professionals and to clarify the roles required for insights production. Of all the applications of the artisan concept, Thompson’s concept of the artisan adds the most value to the Insights Supply Chain by describing a person who can cover the entire production system as a specialist instead of a generalist.

Analytics leadership does not have to choose between the artisan and the data factory. The Insights Supply Chain shows how to design data ecosystems where both models can thrive.