Many organizations could be hesitant to overtake their tech stack and begin from scratch.
Not Notion.
For the three.0 model of its productiveness software program (launched in September), the corporate didn’t hesitate to rebuild from the bottom up; they acknowledged that it was vital, actually, to assist agentic AI at enterprise scale.
Whereas conventional AI-powered workflows contain express, step-by-step directions primarily based on few-shot studying, AI brokers powered by superior reasoning fashions are considerate about instrument definition, can establish and comprehend what instruments they’ve at their disposal and plan subsequent steps.
“Somewhat than attempting to retrofit into what we have been constructing, we needed to play to the strengths of reasoning fashions,” Sarah Sachs, Notion’s head of AI modeling, instructed VentureBeat. “We've rebuilt a brand new structure as a result of workflows are totally different from brokers.”
Re-orchestrating so fashions can work autonomously
Notion has been adopted by 94% of Forbes AI 50 corporations, has 100 million complete customers and counts amongst its prospects OpenAI, Cursor, Figma, Ramp and Vercel.
In a quickly evolving AI panorama, the corporate recognized the necessity to transfer past easier, task-based workflows to goal-oriented reasoning methods that permit brokers to autonomously choose, orchestrate, and execute instruments throughout related environments.
In a short time, reasoning fashions have grow to be “much better” at studying to make use of instruments and comply with chain-of-thought (CoT) directions, Sachs famous. This permits them to be “way more impartial” and make a number of choices inside one agentic workflow. “We rebuilt our AI system to play to that," she mentioned.
From an engineering perspective, this meant changing inflexible prompt-based flows with a unified orchestration mannequin, Sachs defined. This core mannequin is supported by modular sub-agents that search Notion and the net, question and add to databases and edit content material.
Every agent makes use of instruments contextually; for example, they will determine whether or not to look Notion itself, or one other platform like Slack. The mannequin will carry out successive searches till the related data is discovered. It might probably then, for example, convert notes into proposals, create follow-up messages, monitor duties, and spot and make updates in information bases.
In Notion 2.0, the group centered on having AI carry out particular duties, which required them to “suppose exhaustively” about learn how to immediate the mannequin, Sachs famous. Nonetheless, with model 3.0, customers can assign duties to brokers, and brokers can really take motion and carry out a number of duties concurrently.
“We reorchestrated it to be self-selecting on the instruments, slightly than few-shotting, which is explicitly prompting learn how to undergo all these totally different eventualities,” Sachs defined. The intention is to make sure all the things interfaces with AI and that “something you are able to do, your Notion agent can do.”
Bifurcating to isolate hallucinations
Notion’s philosophy of “higher, sooner, cheaper,” drives a steady iteration cycle that balances latency and accuracy by way of fine-tuned vector embeddings and elastic search optimization. Sachs’ group employs a rigorous analysis framework that mixes deterministic checks, vernacular optimization, human-annotated information and LLMs-as-a-judge, with model-based scoring figuring out discrepancies and inaccuracies.
“By bifurcating the analysis, we're capable of establish the place the issues come from, and that helps us isolate pointless hallucinations,” Sachs defined. Additional, making the structure itself easier means it’s simpler to make adjustments as fashions and strategies evolve.
“We optimize latency and parallel considering as a lot as attainable,” which ends up in “manner higher accuracy,” Sachs famous. Fashions are grounded in information from the net and the Notion related workspace.
In the end, Sachs reported, the funding in rebuilding its structure has already offered Notion returns by way of functionality and sooner price of change.
She added, “We’re absolutely open to rebuilding it once more, when the following breakthrough occurs, if we’ve got to.”
Understanding contextual latency
When constructing and fine-tuning fashions, it’s vital to know that latency is subjective: AI should present essentially the most related data, not essentially essentially the most, at the price of velocity.
“You'd be shocked on the other ways prospects are keen to attend for issues and never look ahead to issues,” Sachs mentioned. It makes for an fascinating experiment: How sluggish are you able to go earlier than individuals abandon the mannequin?
With pure navigational search, for example, customers is probably not as affected person; they need solutions near-immediately. “For those who ask, ‘What's two plus two,’ you don't wish to wait on your agent to be looking out in all places in Slack and JIRA,” Sachs identified.
However the longer the time it's given, the extra exhaustive a reasoning agent may be. For example, Notion can carry out 20 minutes of autonomous work throughout lots of of internet sites, recordsdata and different supplies. In these situations, customers are extra keen to attend, Sachs defined; they permit the mannequin to execute within the background whereas they attend to different duties.
“It's a product query,” mentioned Sachs. “How will we set person expectations from the UI? How will we verify person expectations on latency?”
Notion is its largest person
Notion understands the significance of utilizing its personal product — actually, its workers are amongst its largest energy customers.
Sachs defined that groups have energetic sandboxes that generate coaching and analysis information, in addition to a “actually energetic” thumbs-up-thumbs-down person suggestions loop. Customers aren’t shy about saying what they suppose needs to be improved or options they’d prefer to see.
Sachs emphasised that when a person thumbs down an interplay, they’re explicitly giving permission to a human annotator to research that interplay in a manner that de-anonymizes them as a lot as attainable.
“We’re utilizing our personal instrument as an organization all day, each day, and so we get actually quick suggestions loops,” mentioned Sachs. “We’re actually dogfooding our personal product.”
That mentioned, it’s their very own product they’re constructing, Sachs famous, in order that they perceive that they might have goggles on on the subject of high quality and performance. To stability this out, Notion has trusted "very AI-savvy" design companions who’re granted early entry to new capabilities and supply vital suggestions.
Sachs emphasised that that is simply as vital as inside prototyping.
“We're all about experimenting within the open, I believe you get a lot richer suggestions,” mentioned Sachs. “As a result of on the finish of the day, if we simply have a look at how Notion makes use of Notion, we're probably not giving the perfect expertise to our prospects.”
Simply as importantly, steady inside testing permits groups to judge progressions and ensure fashions aren't regressing (when accuracy and efficiency degrades over time). "All the things you're doing stays devoted," Sachs defined. " that your latency is inside bounds."
Many corporations make the error of focusing too intensely on retroactively-focused evans; this makes it tough for them to know how or the place they're enhancing, Sachs identified. Notion considers evals as a "litmus check" of improvement and forward-looking development and evals of observability and regression proofing.
“I believe a giant mistake plenty of corporations make is conflating the 2,” mentioned Sachs. “We use them for each functions; we take into consideration them actually in another way.”
Takeaways from Notion's journey
For enterprises, Notion can function a blueprint for learn how to responsibly and dynamically operationalize agentic AI in a related, permissioned enterprise workspace.
Sach’s takeaways for different tech leaders:
-
Don’t be afraid to rebuild when foundational capabilities change; Notion absolutely re-engineered its structure to align with reasoning-based fashions.
-
Deal with latency as contextual: Optimize per use case, slightly than universally.
-
Floor all outputs in reliable, curated enterprise information to make sure accuracy and belief.
She suggested: “Be keen to make the laborious choices. Be keen to sit down on the high of the frontier, so to talk, on what you're creating to construct the perfect product you possibly can on your prospects.”
[/gpt3]
