The Rise of Analytics Engineering (and What It Actually Changes for Data Teams)
Much (digital) ink has been spilled over the last ten years regarding the collection of techniques and technologies that make up the modern data stack. While there is still no agreement on which warehouse, which EL tool, which BI platform, or which AI copilot (if there even is such a thing) is best, it is agreed that a seismic shift has happened in how data engineering and data analytics is done. Meanwhile, a quieter but more important shift has been happening; the creation of insights has become decoupled from the ingestion and curation of data. Specifically, the Analytics Engineer has become the conduit through which data becomes an asset to be unlocked: a separate function whose whole purpose is to turn raw, chaotic inputs into ready‑to‑use, business‑grade data. If you care about faster insights, less chaos, and actually getting value from AI, that distinction matters a lot.
In late 2018 and early 2019, practitioners (notably within communities like Locally Optimistic) began to use “analytics engineer” to describe this distinct set of responsibilities in documentation, blogs, and job postings. The earliest compressive descriptions I can find for the role appear in pieces authored by Tristan Handy, the founder of dbt Labs, from an article of his in January of 2019, and from Michael Kaminsky, also in January of 2019. Naming the concept proved pivotal in catalyzing the term’s adoption because it encouraged treating analytics transformations as software engineering for data. That shift helped crystallize the role’s identity and vocabulary across data teams.
Starting in 2019, the title started showing up more frequently on job boards, company org charts, industry reports, conference talks, and in tool documentation. By the early 2020s, it became common enough that media outlets and career sites were describing analytics engineering as a distinct role, separate from data engineering or data analysis.
Why Analytics Engineering Exists Now
Everything between (modeling, business logic, data quality, semantics, documentation), was handled on a “best effort” basis by whoever had time. Maybe that worked and maybe it didn’t. Regardless, that world is gone. It’s not unusual for Brooklyn Data to see three or more source systems producing “the truth” and stakeholders who expect self‑serve, real‑time insights. The missing piece here is ownership. Who on the data team is responsible for creating the scaffolding to make data usable. Activities like defining what “customer” means and how to compute it, making sure “revenue” is consistent across Finance, Sales, and Product, keeping transformation logic aligned with how the business works today, not two reorganizations ago, and documenting all of this so people (and AI) can safely reuse it?
The analytics engineer’s job description in one line is to own the transformation layer that makes data genuinely usable.
What Analytics Engineers Really Do All Day
On paper, analytics engineers “build models in the warehouse” using version controlled SQL or another method of data manipulation. In reality, the job is broader and much more strategic. One such activity is turning business concepts into data models: making sure the tables match how the business thinks and talks. This can also be described as creating a semantic model.
Part of creating an effective semantic model is centralizing knowledge in version-controlled files and not having important information live in an analyst’s personal notebook, a “don’t touch this” tab in a spreadsheet, or a slide somewhere in last year’s QBR deck. Business rules become code with owners, history, and review.
Another important responsibility for Analytics Engineers is owning the core data quality testing. Tests ought to live near the transformations themselves so that bad data doesn’t have a chance to be propagated through to schema and used by the end user.
Finally, Analytics Engineers document the context of the data they model. Either through a data catalog, data dictionary, or some other discoverability tool. Analysts, business stakeholders, and AI tools should never be guessing about lineage, definition, or type.
The Quiet Win: Freeing Data Engineering and Enabling Analysts
All of this raises an obvious question: what are data engineers doing while analytics engineers are busy shaping the semantic layer? The short answer is exactly what you want them to be doing. Data engineering is increasingly a platform, reliability, and performance role overseeing data ingestion and pipeline health, storage patterns, compute optimization, cost control, security, permissions, governance, and observability, SLAs, and handling incident response.
These problems are not getting easier. Offloading modeling and business logic to analytics engineering lets data engineers focus on building and running a robust platform.
The result is a cleaner layering and more differentiated roles and responsibilities in your data team:
- Data engineers: make data arrive - reliably, securely, at scale.
- Analytics engineers: make data usable - modeled, tested, documented.
- Analysts and business users - turn that data into decisions, strategies, and experiments.
When you get that separation right, lead times shrink dramatically. New questions don’t require “new pipeline plus three ad‑hoc joins.” They’re often just another query against a well‑designed model.
Where Analytics Engineering Meets AI
Now add generative AI to the mix. Many firms are adopting AI and tools promise natural language queries over your warehouse, AI copilots sitting inside your BI tool, and agentic systems that “go run the analysis”. Point any of those at a swamp of semi‑modeled tables and you get wrong answers delivered with confidence and no visibility, inconsistent metrics, and hallucinated interpretations of poorly documented fields. Technology will keep improving and performance will follow, but it will not solve the core problem: models cannot infer your business logic from a pile of raw tables.
Analytics engineers are the human in the loop. They ensure transformation logic lines up with real‑world processes, encode nuances (edge cases, exceptions, historical changes) as explicit logic, define tests so AI isn’t building on obviously broken data, and enrich models with descriptions and metadata that LLMs can actually use to deliver reliably accurate results. “AI‑ready” data is not just clean—it’s modeled, tested, and explained. That’s analytics engineering plain and simple.
Adapting Your Org Design
Here are a few best practices when building your data org:
- Treat analytics engineering as a first‑class role, not a side quest.
Hire for it explicitly. Give it a clearly defined scope: the transformation layer and semantic model. - Clarify ownership.
Data engineering owns ingestion and platform.
Analytics engineering owns business logic and modeled layers.
Analytics owns insights, experimentation, and stakeholder relationships. - Invest in workflows, not just tools.
Version control, code review, CI for your models, and a test suite that reflects what your organization actually cares about. - Measure success in business outcomes.
- Time from new product launch to reliable metrics
- Fewer metric debates in leadership meetings
- Faster experimentation cycles
- AI features powered by trusted data, not one‑off, hand‑curated extracts
The Takeaway
Most organizations don’t need more dashboards or another AI pilot. They need a reliable way to get from “we collected the data” to “we can use this confidently, at scale, across humans and machines.”
Analytics engineering unlocks the value of data by turning it into a consumption-ready asset. It turns raw feeds into data products, embeds business logic where it belongs, catches issues early, and gives both people and AI systems something solid to stand on. It also lets data engineers focus on building a healthy platform instead of fighting over who owns the definition of ARR or what a customer is. In other words: if you’re serious about making data and AI a core part of how your business operates, analytics engineering isn’t a niche specialization; it’s the glue that keeps your data project together.