Olivier Pomel

The through-line

Pomel's recurring argument is that the hard part of building software tools is never the technology on its own, it is knowing what problem is actually worth solving and then earning the right to be trusted with it. He traces Datadog's origin not to cloud but to a human problem, developers and operations "fighting all the time" in separate tools and separate worlds, with cloud as the entry point rather than the thesis [3][4]. The same instinct shows up in how he talks about competing from New York against better funded, better pedigreed Bay Area rivals: the advantage was proximity to ordinary companies and "having our customers keep us honest" rather than being "blinded by everything else that was going on around us in the ecosystem" [1]. And it shows up in his allergy to AI marketing, formed by a decade of watching a category sold as AI that was really "rules with regular expressions" [3].

What has shifted is that the technology has finally caught up with the claims, and that has forced a change in method. For fifteen years Datadog could rely on "very very clear customer demand" to tell it what to build; with AI, customers say AI should do this for them but "the that and how and what good enough means" is unclear, so the company now has to "build more speculatively" and get ahead of what the market will bear [3]. Three or four years ago being reliably right about root cause was "science fiction" and now it is within reach [2], but he is careful that optimism does not curdle into overselling again.

On starting from the customer's problem rather than the ecosystem's consensus

Pomel is blunt that Datadog's early edge was informational, not technical. The competitors were all Bay Area based, all better funded, with founders who had built similar companies and worked at hyperscalers; what Datadog had instead was fear of getting it wrong, which pushed the team to talk to every company that would listen and to find the right problem first [1]. He extends this into a broader claim about geography: it is "actually probably easier to understand how to build a right product in New York than it is in the Bay Area", because New York is a normal city with normal companies, whereas in an environment where everyone is in tech, an idea you agree with may simply be something you said three days ago that has circled back to you through several people [1].

That same distance from consensus shapes how he reads sentiment about AI itself. He describes a gradient running from the West Coast to Europe: near San Francisco the view is that we have barely two years to find a hobby before the machines do everything, while in France people are culturally more cynical and will tell you large language models may revolutionise poetry but not much else. "Of course the truth is somewhere in the middle" [1]. His general advice to founders follows from the same reasoning about position: when the stack needed to deliver value is very thick, "it's much better to be on the ends of it", close to the problem or the customer, rather than "in the middle of the sandwich" where your fate is far harder to predict [3].

On bringing dev and ops into the same place

The founding pain was that developers and operations lived in separate worlds with separate tools and spent the day finger pointing, and the goal was to get them into the same application speaking the same language [3][4]. Cloud turned out to be the insertion point because succeeding in the cloud genuinely required collapsing two teams and a long shipping cycle into something tighter, and much of the savings from migrating came from exactly that collapse [3]. Observability itself he treats as the modern name for monitoring, a category that formed when infrastructure monitoring, application performance monitoring, log management and network monitoring, each its own microcosm, converged into something end to end running from servers and networks through to what end users are doing and what it means for the business [2]. He is not fond of the word: it sounds "reductive and passive" when much of the work is anything but [2]. He similarly rejects the idea that metrics, logs and traces are three separate pillars, comparing them to particle wave duality, different views of the same thing that should live in one place; what matters is capturing as many inputs and outputs from as many sources as possible [4].

The lessons transfer directly to today's mix of AI engineers, AI product managers and non-technical people building agents: put them in the same application, give them "the same reality in front of you" instead of tools that are each right in their own way, plug into the workflow people actually have rather than expecting them to show up because the tool exists, and above all "you have to minimize friction" and show value fast [4]. Pricing is part of that design. Datadog does not sell per seat, because per-seat pricing makes customers optimise by keeping people out of the product, which is counterproductive when the entire point is to bring as many teams together as possible [4]. He does note the limit: mixing very technical and less technical users means things technical users expect will go unused by others, so that has to be handled carefully [4].

On refusing to oversell AI

Pomel entered the market when AI was "a bit of a curse word" in systems management, after decades of vendors promising the computer would solve everything [3]. There was even a Gartner category, AIOps, with a quadrant and winners, in which he says none of the products contained any AI: what was called AI was "rules with regular expressions", occasionally with some additions and divisions [3]. He puts it more sharply elsewhere as "AI better known as multiplications additions and subtractions" [1]. So early on the company decided not to plaster its website with claims of magical AI, while quietly doing a great deal of classical machine learning in the back end to decide what data to keep, what to surface and how to detect anomalies, work you simply have to do for observability at scale [3].

He thinks the claims are now defensible, but the discipline still matters: it is "very very important" and very hard "not to oversell on AI", because the industry has produced generation after generation of products that promised AI and underdelivered [1]. His stated worry is a market cool-down where the technology does not work as well as people imagined, everyone is disappointed, and you end up having to stop saying AI at all. His guess is it probably will not happen because delivery is keeping pace, but the hedge is deliberate [3]. He is equally happy to puncture the technology at his own expense, noting that ChatGPT describes Watchdog as good old fashioned AI and also insists he is a snowboarding champion, having never been on a snowboard [2].

On the three layers of AI opportunity

He maps the opportunity in three levels. The first and, in his words, most straightforward and boring is that more applications are being built with AI and those applications have a different shape, more GPUs and far more data, which shows up immediately as infrastructure consumption from model builders and the companies powering agents [2]. The second is the growing set of applications built on top of models, which are not deterministic and which you build, manage and understand in a fairly different way, an area Datadog serves with LLM Observability [2]. The third is turning the new capability inward: using AI so that engineers do not have to solve issues themselves, so the machine detects more and resolves more [2]. Datadog is investing in all three, and he frames the audience sizes as an inverted pyramid, with a small number of model builders, more companies building on models, and eventually every company in the world wanting automation, since "the dream is you never have to wake up in the middle of the night ever again to fix an issue" [2]. The reason the largest immediate needs sit with the model builders is simply that they are upstream of everyone else's consumption of AI [2]. That tier is already material to the business: AI native customers were disclosed as between three and four percent of revenue and growing very fast [1].

On trust, precision and the false-positive lie

His most emphatic operating lesson is about what customers say versus what they do. Customers will tell you they would rather receive false positives and judge for themselves, and Pomel calls that a lie: "you send two false positives to people in a row and then they turn you off forever" [2]. People will chase down the wrong path for a human coworker and not for a machine, so "the bar for precision needs to be very high" [2]. The naive approach fails because "the LLMs in particular are not good at understanding when they know and what they don't know. They'll always gladly answer", and might be wrong a lot of the time [2]. Much of what Datadog builds is therefore about knowing when it is right, so it can volunteer an answer or fix something directly only when the odds are very high. The market entry strategy follows: "you can get higher precision by having lower recall", starting with a narrow subsection of incidents where confidence is greatest [2]. He calls being an incumbent that already holds the data a real luxury here, since you can pick which cases to automate, an option a self-driving car does not have, because it cannot decide to only make left turns [2].

Tolerance varies by domain. Users accept automated decisions more readily in security than in operations, because the risk reward differs: "it's okay to crash a workload if you avoid a security incident", and less okay to crash a workload in order to avoid crashing a workload [2]. The constraints differ too. Root cause work needs breadth across all possible causes and very low latency, since taking longer than the humans makes the answer useless; security analysis has hours to work with, but the stochastic behaviour of an agent is deeply unsettling if the same signal is called benign once and dangerous the next time, so stability of decisions matters as much as precision [3]. He also pushes back on the idea that any of this is easy work for AI: self-driving does something almost any human over sixteen does without thinking, whereas understanding an incident takes large teams of highly qualified people and sometimes weeks of thought afterwards [2]. And he draws a direct lesson from the cloud migration, where Amazon handled the security fear extremely well from day one, arguing the same needs to happen around AI models so the main fears come off the table and everyone can build with confidence [2].

On form factor and agents

Pomel treats interface design as an open research question that sets user expectations. Datadog's first AI build was a chatbot like everyone else's, and the finding was that "people are not quite sure what to ask" when the problem they face is not naturally a prompt, which he found disarming, and which echoed earlier waves of chatbots that emerged, disappointed and disappeared [1]. His verdict is that "the chat interface was a great start" that opened everyone's imagination but "it's not the be all end all", and that most AI functionality will not run through chat in the long run [2]. He observes that ChatGPT's fastest early wins were exactly the prompt-shaped use cases, homework being literally a prompt, and that most industry use cases are not prompts [1].

The successor is an autonomous agent for incident resolution: you get paged in the night, you join the chat room with your team, and the agent joins too, reporting the experiments it has already run, pointing at the database, noting that the database team has not been paged and offering to page them, flagging customer impact and offering to declare an incident and update the status page [1]. It was trained and tested in simulation against a collection of past incidents and rolled out internally and with design partners before the real world [1]. The stated ambition is that over time it solves things on its own, but he says the most important part right now is experimenting with the form factor and finding the natural way for users to start building trust, always the hardest part of an AI product [1]. Under the Bits AI brand there are several specialised agents with their own training and evaluation data that can communicate with each other, covering triage in on-call and outage situations, fixing errors detected in production, optimising cost and performance and identifying root cause, alongside a separate agent that judges whether security signals such as simultaneous logins to different applications are real or benign [2][3]. He will not predict what application form factors look like in five or ten years, saying the boundary between language, UI, proactive and reactive is still unsettled [3].

On evals as the real work

Pomel calls evals "the single most important part" of AI functionality and agent development [4]. Datadog's investigative agent stalled on quality until the team got good, appropriate and reputable evals in place, after which progress accelerated sharply, a before and after he says was clearly trackable; the first step for anything new shipped on that product is now building the evals for it [4]. They run at the level of individual steps and end to end, and the hardest and most valuable component is the data set: selecting, annotating and cleaning the data is a very big job, and Datadog's advantage is being a SaaS business plugged into the exact workflows it is trying to automate, which gives it a rich source to draw from [4]. The teams that build the products write the evals, but the data sets are assembled and annotated through company-wide programmes, because the agents touch every product and data source and so every team has to contribute [4]. Old eval cohorts keep running alongside new ones as a discipline against self-flattery: "we don't choose the data that makes our products look better" [4]. He distinguishes evals from tests, which at least gave a pass or fail; an eval is not black and white and is more open to interpretation about whether it is even measuring the right thing [4]. On the market, he expects frontier labs to keep everything homegrown as core value, while tier two AI native companies and the broader enterprise world will want off-the-shelf tooling with clarity on how things get built, even though many enterprises have not yet reached the point of needing it [4].

On data quality as the actual moat

Toto, named for the dog in the Wizard of Oz, is Datadog's transformer-based time series model, built internally on the company's own observability data with a purpose-built observability benchmark, predicting how numerical series will move over the next minute, hour, day or month and feeding anomaly detection by comparing prediction against reality [2][3]. What shocked the team was that this very first model came out "state-of-the-art for all of time series", better at weather prediction than models built for non-observability data [3]. His reading is that beyond having good people, "the quality of the data makes everything": Datadog ingests billions of records per second, and crucially it has strong signals about which data matters, knowing what actually wakes people up at night and what they look at when investigating, so those signals can be weighted during training [3].

On complexity migrating from dev into production

He rejects the premise that traditional observability deals in clear black-and-white failures: "mostly they are not" [4]. The reality is systems that are individually fine and broken in combination, strange degradations, a world that works for 99.9% of users and is horrible for the rest, and correctness issues that are hard to detect and expensive for the business [4]. His model of complexity is historical and unsentimental. Every generation of productivity improvement, higher-level languages, libraries, open source, SaaS, cloud, buys speed at the cost of understanding, and "to me, the complexity is all the stuff that's there that you don't really understand" [4]. Fifty years ago on punch cards you hand coded every bit and knew exactly what was happening while being wildly unproductive; now you call things and have no idea how they work until something changes and it breaks [4].

AI accelerates the same trade. With a model-based application you do not spend months constraining every use case and writing a spec ahead of time, and while you obviously do not "just install it on prod and yolo", you end up doing the knowing-it in production, which shifts time and value out of dev and into running the thing [4]. The same holds for AI-written code: you wrote it faster, you proofread it and ran it through internal process, but you spent less time on it and understand it less, so you have to be more careful in production [4]. He points to exploding demand for code security and generally more demand for tooling that looks at the code as concrete evidence of that shift [4]. He also notes the specifically new components: upstream model providers change quality and functionality under you, and tool calling extends the dependency further, which he views as a sharper version of an old problem with upstream services [4].

On business model and market structure

There are two reasons to price on consumption: alignment with the value the customer gets, and a heavy infrastructure cost component, which is why cloud providers charge that way on relatively lower-margin storage and compute [1]. AI today is expensive, so consumption pricing has a real cost-protection function, though he thinks functionality that is expensive now may be much cheaper in two to four years, at which point the cost of AI could resemble the cost of operating ordinary software [1]. The deeper question he raises is that non-consumption software is usually sold per seat, and per-seat pricing gets tricky when the product may inherently be replacing seats, so the way you sell for value has to change; he treats the business model of AI as genuinely open [1].

On market shape, he is wary of the assumption that one player takes everything. There was a period, and many a re:Invent, when it seemed plausible that in fifteen years everyone would work for Amazon, which worried him because a monoculture leaves less value to add; instead there is now a healthy race between three players with more to come [1][3]. He explicitly declines to map foundation models onto cloud vendors, because there are extra variables beyond who dominates, chiefly the delivery mode: run it yourself, open source, behind an API, or operated for you [1]. His analogy is the database market, which splits healthily across open source you run, open source someone runs for you, proprietary you run, proprietary run for you, and services behind an API where you may not even know a database is involved. Companies use a mix, running their own MySQL or Postgres while paying for MongoDB behind an API and consuming DynamoDB or BigQuery, and his best guess is the LLM market ends up looking similar [1].

On building AI inside a company that already works

The push came partly from the existing machine learning team moving into the next generation of the technology, partly from elsewhere, and he describes both a bottom-up and a top-down element [3]. ChatGPT was the hinge: suddenly natural language data that had been effectively off limits became usable, reasoning and stitching together steps of a process became conceivable, and transformers could be extended to numerical work more efficiently than classical machine learning allowed [3]. That work now splits into an applied AI side, taking known research and tuning it for Datadog's use cases with engineering teams building products on top, and a separate AI research team doing novel work [3]. The reason for the research team is a change in tempo he has lived through personally: when he started his career in research it took ten to fifteen years for research to become products and most of it turned into patents rather than applications, whereas at the frontier of language models and reasoning "the research is turned into products within six months and it's not published", which means you cannot rely on the public literature and have to generate your own [3].

He is also candid about the cognitive difficulty of building here. The mental model everyone trained for fifteen years stopped working overnight: conversing with a machine that speaks better than any human was impossible and then became the easiest thing in the world, which warped the industry's sense of what is hard [3]. Because "the machine is so good at presenting the results", people assume the back-end processing is equally good, which is not yet the case [3]. And working with unpredictable systems is "a joy and a nightmare", since evaluation becomes the hardest part [2]. Compounding it, the stack itself is unsettled: the world changes between starting to build and shipping, things that looked workable stop working and things that looked out of reach become possible, so fast iteration is the only option for Datadog and everyone else [3][1].

On why enterprise AI adoption will take longer than expected

Pomel splits customers in two. A small number are AI native, mostly infrastructure builders working on GPU management, foundation models across text, image, video and sound, and vector databases; they know exactly what they are doing and their concern is scaling to serve their own customers [1]. The vast majority, from small business through mid-market to enterprise, are experimenting but very few are in production and fewer still at scale [1]. Their challenge is productionisation: understanding how to deploy models safely and what the right form factors for AI applications actually are [1]. His worry is that this takes longer than expected, because of lag in people rather than technology, and because the underlying stack, databases and models alike, is changing fast enough to unsettle anyone building on it [1]. His analogy is the iPhone, where the applications we now use took years to emerge, not because the hardware needed to change but because it took time to find the right form factors and for behaviour to shift [1].

Takeaways

• Customers say they want false positives so they can judge for themselves, and that is a lie: "you send two false positives to people in a row and then they turn you off forever", so precision, not recall, sets the bar for automated observability [2].
• Enter an AI market by deliberately trading recall for precision, starting with the narrow set of cases where confidence is highest, an option available to an incumbent that already holds the customer's data [2].
• Evals are "the single most important part" of agent development, and the hardest part of evals is assembling, annotating and cleaning the data set, which at Datadog is a company-wide programme rather than a single team's job [4].
• Datadog's Toto time series model came out state-of-the-art across all time series, including weather, which Pomel attributes to volume plus knowing which signals matter: "the quality of the data makes everything" [3].
• Do not sell per seat if the goal is to get multiple teams collaborating in one product, because per-seat pricing makes customers optimise by keeping people out [4].
• Chat was a good start but is not the end state; most AI functionality will not be delivered through a chat interface, and getting the form factor right is how trust gets built [1][2].
• AI-written code and model-based applications shift work from design time to run time: you understand the system less, so you must observe production more, which is showing up as exploding demand for code security [4].
• When a stack is very thick, build at its ends, close to the customer or the problem, rather than in the middle of the sandwich where outcomes are hardest to predict [3].