← Back
Google Research

Does better work always mean better workers?

•5 min read•
#llm
Does better work always mean better workers?
Level:Advanced
For:AI Engineers
✦TL;DR

A three‑month randomized field experiment with 133 patent attorneys showed that routine use of Google Labs’ AI patent‑writing assistant (now part of Gemini Notebook) lifts drafting quality by 0.34 SD after 10 days and 0.38 SD after 90 days, equivalent to roughly a 10‑ to 11‑percentile point gain. Senior lawyers who used the tool for 90 days improved their judgment scores, while junior lawyers showed no average skill gain, with scores polarizing toward higher and lower extremes. The findings suggest that sustained AI assistance can boost short‑term productivity but may have divergent effects on long‑term expertise depending on seniority.

⚡ Key Takeaways

  • AI tool access raised drafting scores by 0.34 SD (≈10 percentile points) after 10 days and 0.38 SD (≈11 percentile points) after 90 days.
  • The experiment randomized early access to Google Labs’ AI patent‑writing assistant (now Gemini Notebook) among two‑thirds of lawyers at each firm.
  • Senior attorneys exhibited stronger judgment after 90 days of AI use, whereas junior attorneys showed no average skill gain and a split between stronger and weaker scores.
  • Evaluation used blinded grading by independent legal experts on a Likert‑scale rubric covering five quality dimensions.
  • The study required (1) sustained AI usage embedded in ordinary work, (2) a credible unassisted judgment benchmark, and (3) blinded expert grading.

🔧 Tools & Libraries

Gemini Notebook (Google Labs AI patent writing assistant)
💡 Why It Matters

Engineers building AI‑augmented professional tools must account for user seniority, as AI can amplify productivity while potentially reinforcing or eroding expertise differently across experience levels.

✅ Practical Steps

  1. Deploy the Google Labs AI patent‑writing assistant (Gemini Notebook) in a controlled pilot, granting access to a majority of users while withholding it from a control group.
  2. After 10 days and again after 90 days, have all participants draft patents from standardized inventor materials and collect blinded expert ratings using a Likert‑scale rubric.
  3. Analyze performance changes in standard deviation and percentile terms, and segment results by seniority to assess differential skill impacts.

Want the full story? Read the original article.

Read on Google Research ↗

More like this

Censored vs. Truncated Regression in Python, Explained

Towards Data Science•#llm

How Google Duplex evolved to also call businesses for you in Gemini

Google Research•#agents

Q&A: Enabling more human-like interactions and understanding in multimodal foundation models

MIT News AI•#llm

Welcome Inkling 2 Mini, Cutting-edge Performance for your Local Setup

Hugging Face Blog•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING