Does better work always mean better workers?
A three‑month randomized field experiment with 133 patent attorneys showed that routine use of Google Labs’ AI patent‑writing assistant (now part of Gemini Notebook) lifts drafting quality by 0.34 SD after 10 days and 0.38 SD after 90 days, equivalent to roughly a 10‑ to 11‑percentile point gain. Senior lawyers who used the tool for 90 days improved their judgment scores, while junior lawyers showed no average skill gain, with scores polarizing toward higher and lower extremes. The findings suggest that sustained AI assistance can boost short‑term productivity but may have divergent effects on long‑term expertise depending on seniority.
⚡ Key Takeaways
- AI tool access raised drafting scores by 0.34 SD (≈10 percentile points) after 10 days and 0.38 SD (≈11 percentile points) after 90 days.
- The experiment randomized early access to Google Labs’ AI patent‑writing assistant (now Gemini Notebook) among two‑thirds of lawyers at each firm.
- Senior attorneys exhibited stronger judgment after 90 days of AI use, whereas junior attorneys showed no average skill gain and a split between stronger and weaker scores.
- Evaluation used blinded grading by independent legal experts on a Likert‑scale rubric covering five quality dimensions.
- The study required (1) sustained AI usage embedded in ordinary work, (2) a credible unassisted judgment benchmark, and (3) blinded expert grading.
🔧 Tools & Libraries
Engineers building AI‑augmented professional tools must account for user seniority, as AI can amplify productivity while potentially reinforcing or eroding expertise differently across experience levels.
✅ Practical Steps
- Deploy the Google Labs AI patent‑writing assistant (Gemini Notebook) in a controlled pilot, granting access to a majority of users while withholding it from a control group.
- After 10 days and again after 90 days, have all participants draft patents from standardized inventor materials and collect blinded expert ratings using a Likert‑scale rubric.
- Analyze performance changes in standard deviation and percentile terms, and segment results by seniority to assess differential skill impacts.
Want the full story? Read the original article.
Read on Google Research ↗