How the Bureau of Labor Statistics is Ditching Hand-coding Data

How the Bureau of Labor Statistics is Ditching Hand-coding Data

Implementing machine learning and artificial intelligence has radically transformed the Bureau of Labor Statistics’ productivity, freed up its workforce to perform less menial tasks and has resulted in more accurate survey analysis.


The fact-finding agency processes hundreds of thousands of surveys each year to provide the government and public with essential statistical data about society and the economy. In the past, converting the text-heavy records into different codes that make sense of the data required BLS workers to engage in tedious manual labor that didn’t always result in the most accurate outcomes. But automating the once manual processes has had lasting impacts across BLS.


“This all actually worked out much better than we expected,” Alex Measure told Nextgov. Measure was originally hired as an economist at BLS, but over the last eight years he’s led some of the agency’s efforts around integrating machine learning to complete tasks previously done by hand.


For example, each year the agency conducts the Survey of Occupational Injuries and Illnesses, which collects hundreds of thousands of written descriptions regarding work-related afflictions. In the past, Measure said humans would spend countless hours converting the key pieces of text from each survey into codes so that BLS could discern the data. The results would provide insights like how many U.S. janitors were injured on the job annually, or what the most common injuries might be.


“As you can imagine, when you are collecting about 300,000 of these each year and having people read through them by hand and code them by hand, it really adds up to being a lot of work,” he said.


The bureau began exploring different ways to use computers to aut ..