HARRISBURG — When Pennsylvania announced plans in 2024 for a first-of-its-kind generative artificial intelligence pilot program with the tech giant OpenAI, Steve Catanese believed he was seeing a familiar pattern: the introduction of a new technology that could jeopardize the jobs of his union members.
As president of the labor union SEIU Local 668, which represents 10,000 state workers, Catanese approached Pennsylvania officials about entering into an agreement to provide its members with job protections as the state continued to introduce AI tools.
The state agreed to talk. And, to the pleasant surprise of Catanese, a series of back-and-forth discussions throughout that winter led to a deal. Gov. Josh Shapiro published a public letter outlining commitments to state employees amid the rising popularity of AI use, and the union reached a separate agreement with the state, establishing protections for its members.
Across Pennsylvania’s government, more than 3,000 state employees are now using AI tools, up from the 175 who first piloted the technology. And 6,000 more are currently enrolled in training programs as a prerequisite for accessing the software. In June, Code for America — a civic tech nonprofit — recognized Pennsylvania as one of the top states for AI readiness.
The state, which has established a Generative AI Governing Board and a series of policies regulating its AI use, has stressed that using AI tools has improved government efficiency without eliminating jobs.
Bry Pardoe, the state’s chief information officer, said in a statement that the Shapiro administration is “focused on the safe and responsible adoption of generative AI and other AI technologies.”
Today, Pennsylvania state employees use AI for a range of functions but focus on relatively basic tasks: deciphering handwritten clemency applications, scanning documents, and triaging customer service requests.
Most employees access generative AI tools through no additional cost according to the Office of Administration. For the 721 employees who have access to ChatGPT Enterprise, the cost is $25 dollars per month for each user, bringing the annual cost — based on current usage — to $216,000 a year.
Mark Fagan, a lecturer at Harvard University’s Kennedy School of Government and author of a book on how governments can use these tools, said that generative AI can serve as an important resource to deliver public services to constituents.
“There are endless opportunities for using AI to improve the way we either provide higher quality, more efficiency, or more equity in our public services,” he said.
But to do this best, he said, governments should first find friction points and inefficiencies in their operations and then determine if the technology can help.
“AI is often a solution to pain, but that’s not enough,” Fagan said. “If you’re going to use AI, it should be the best solution to the problem.”
A first-of-its-kind pilot
In January 2024, the Shapiro administration launched a pilot program in partnership with OpenAI, the company behind the generative AI chatbot ChatGPT, to test the rollout of tools among a select group of executive agency employees.
The program made Pennsylvania the first state in the nation to enter into an agreement with OpenAI, and also marked the first time employees within the state were permitted to use generative AI tools.
“The pilot was designed to explore using AI tools from the perspective of Commonwealth employees; to understand when, where, and how tools could be of useful to them, where using these tools may introduce opportunities for risk, and areas where employees might need additional training or resources to be successful,” Dan Egan, a spokesperson for the Office of Administration, which has overseen the state’s AI rollout said.
Sam Altman, OpenAI’s chief executive officer, said at the time of the rollout that the pilot “showcases Pennsylvania’s commitment to innovation” and would “provide valuable insights into how AI tools can responsibly enhance state services.”
(OpenAI has since introduced a new model tailored specifically for government use and entered into partnerships with other states across the country. In February, Massachusetts officials announced they would make ChatGPT available to their entire 40,000 person executive branch.)
In Pennsylvania, the pilot involved 175 employees from 14 state agencies.
Participants, including Office of Administration employees, lawyers, and construction project managers, were equipped with access to ChatGPT Enterprise, OpenAI’s model for businesses which contains additional privacy features. Information submitted to the chatbot isn’t used to train additional models.
Employees most commonly used ChatGPT during the pilot to assist with drafting emails or other writing.
With access to ChatGPT, participants reported saving an average 95 minutes per day and 85% of employees said their experience was “somewhat” or “very positive.”
During a news conference last March, Shapiro said state employees from lawyers to construction managers used the tools and found them beneficial. The pilot, the first-term Democratic governor added, allowed his administration to reduce the time needed to onboard a new employee by 30 days and to help to consolidate the number of state technology policies by 60%.
“There are too many public officials who think they can stick their head in the sand and ignore this,” Shapiro said. “I believe we need to embrace them and we need to use it for the betterment of society and the betterment of the people we serve.”
Protections for employees
Shapiro, in his letter to state employees, wrote that his administration would implement AI tools with a goal of boosting productivity “not with the agenda of substituting Commonwealth employees with AI.”
In its agreement with SEIU, the state made a commitment that AI tools will follow a “human in the loop” process in which “human workers oversee, review, and guide generative AI tool outputs to ensure accuracy, fairness, and compliance with workplace standards.”
The agreement also affirmed the definition of a public employee as a person, something Catanese said was particularly important to his union.
“I think right now, whether you work for the government or big business, or whatever type of entity, you want to work for an employer that actually sees in the long term there is a role for humans,” Catanese added.
But, he is cognizant that the technology is advancing rapidly, and the union will need to remain vigilant in ensuring those human-in-the-loop protections remain in place to protect workers.
Chinmayi Sharma, a professor of law at Fordham Law School whose research focuses on artificial intelligence, said a dissonance can exist between the draw of using AI tools and the diligence required to check their work.
Organizations may turn to AI tools to make jobs more efficient or more accurate. But ensuring there is a human involved to check the work of those tools requires an additional level of vigilance and care, she said.
“Are these humans in the loop being trained about AI to the point where they can identify subtle or nonobvious errors?” Sharma, whose research is generally cautionary about the use of AI, said.
As far as he knows, Pennsylvania is the only state government at this point that has given a union collectively bargained protections around the use of AI, Catanese said.
Within the union’s membership, perspectives on the rapidly changing technology are varied.
“Most of our members do have some serious concerns about what it means for their future in the long term,” Catanese added.
Other members, Catanese said, believe the tools could help lessen their workload. He said current use cases and restrictions on the types of data that can be entered into the tools means that some of those benefits aren’t yet clear.
“Is it fully guaranteed forever? No,” Catanese said of the protections agreed to by the state. “Is it some level of protection that almost every public servant in the country doesn’t have other than our members at the state? Yes.”
‘A solution’
The state’s use of AI is governed by a set of policies that are reviewed annually.
Human involvement is required in work produced by generative AI and any use of AI tools must be disclosed if a customer is directly interfacing with them. Personally identifying, confidential and nonpublic information also can’t be submitted into AI interfaces.
Throughout the AI rollout, the state has consulted researchers through ongoing partnerships with Carnegie Mellon University and the University of Pennsylvania. Subject matter experts from both institutions have answered questions for the state on an as-needed basis as the administration decides to implement new tools and policies.
As the state considers future use cases, Pardoe said she wants “people to have confidence in the services they get from state government.”
That means, she said, “being thoughtful and deliberate about where and how we choose to implement AI as a solution.”
As AI becomes more complex, Sharma predicted that the same logic that makes AI tools attractive now will continue to apply as the technology becomes more advanced. If a government is under-resourced, temptations could exist to redirect resources away from paying human employees and toward something else.
And, she noted that officials may be motivated to continue to adopt more tools.
“I don’t know of a politician that’s going to enjoy coming out and saying, ‘We just put a lot of time and effort into this AI pilot, turns out we were wrong, it’s not a great thing for us to do and we are rolling it back,’” she added. “That’s a tough sell.”
Fagan, the Harvard lecturer, believes that states with successful pilot programs, like Pennsylvania, will likely adopt the technology at a faster rate in the future.
The important question, he said, won’t be whether more places are using these tools, or whether existing states expand the number of employees using them.
“The question is whether it creates value or not,” he said of government AI usage. “The real value is in finding problems, finding where there is pain, and solving it.”
Ethan Young is an intern with the Pennsylvania Legislative Correspondents’ Association. He can be reached at ethanmyoung@gmail.com.
