Over the past year we’ve been exploring how AI agents might support everyday work across Jisc. Our early pilots have focused on practical use cases where agents could help staff find information, answer questions and reduce repetitive tasks.
This post builds on our earlier blogs exploring the outcomes of Jisc’s internal AI impact workshops and the practical themes emerging from AI adoption across the organisation. Here, we’re focusing on what happened when we moved from identifying opportunities for AI to building and testing agents in practice.
Where AI fits: insights from Jisc’s internal AI impact workshop
AI in practice: exploring five key themes for AI adoption at Jisc
In many ways, building the agents was the easy part. What became clear quite quickly was that the harder challenge was understanding how those agents could be shared, scaled and sustained across an organisation.
We’re sharing our experience because we hope it will be useful for members across the Further and Higher education sectors who are beginning similar work, or finding themselves getting stuck for reasons that are not always obvious.
Following our internal AI impact workshops, we began developing agents to support areas including our HR, Legal and Procurement teams. The aim was simple: help colleagues find information more quickly, navigate guidance more easily, and reduce the time spent answering repeated questions. Building the first agents was relatively straightforward but moving from a working prototype to an organisation-wide service was not.
During our early pilots, much of the discussion centred on what we called “tokens”. Microsoft now refers to these as Copilot Credits, so we’ll use that terminology throughout this blog.
It worked for us… but not for everyone else
As we moved from experimentation into live pilots, we started exploring how agents could be used in everyday work. We wanted to connect agents to SharePoint content, help staff find information more quickly and make those tools available across teams.
The first agents worked well during development and testing. However, as we started sharing them more widely, we ran into challenges that we hadn’t anticipated.
For a while, we thought we were dealing with a technical problem. After looking into it, we realised the issue wasn’t the agent itself. It came down to licensing and how different types of agents can be shared.
Two ways of building agents
As our pilots progressed, we started to understand how important the distinction between Microsoft 365 Copilot and Copilot Studio would become.
Microsoft 365 Copilot is primarily designed to support individual productivity. Users with a Microsoft 365 Copilot licence can create and use agents, and in some circumstances share them with other colleagues.
Copilot Studio is designed for organisation-wide deployment. It allows organisations to build agents that can be made available much more broadly, including to colleagues who do not have a Microsoft 365 Copilot licence themselves. Usage is measured through Copilot Credits rather than being tied solely to individual licences.
This distinction became particularly important in our pilots with HR and Procurement, where the aim was to support the whole organisation rather than a small group of early adopters.
In one of our pilots, an agent connected to SharePoint content worked perfectly during development and testing. Once we attempted to make it available more widely, colleagues without a Microsoft 365 Copilot licence could not access it.
The issue was not permissions, configuration or the quality of the agent. We had focused on getting the agent working, without fully considering how it would be made available to a wider group of colleagues.
Which approach is right for your use case?
| Question | M365 Copilot | Copilot Studio |
|---|---|---|
| Can I create an agent? | Yes | Yes |
| Is it suitable for testing and small groups? | Yes | Yes |
| Can I deploy it across my organisation? | Limited | Yes |
| Can I publish it to multiple channels? | No | Yes |
| Is usage measured using Copilot Credits? | No | Yes |
Understanding Copilot Credits
Once we understood the licensing model, another issue became clear. Copilot Studio uses Copilot Credits to measure usage. Credits are consumed when people interact with an agent, whether that’s asking questions, retrieving information or generating responses.
A Copilot Credit pack provides 25,000 Copilot Credits per month. What we didn’t know, and are still learning about, was how quickly those Credits could be consumed once agents were made available to larger groups of staff.
For us, a Copilot Credit pack costs around £150 per month and this gives us 25,000 Credits to use across all of our agents. We also explored pay-as-you-go pricing, where each Copilot Credit costs around one penny. To give an example of usage, in our HR pilot, 775 questions used 1,137 Credits, which would have cost around £11.37 on a pay-as-you-go basis.
That gave us a useful starting point, but it also highlighted how difficult it is to estimate usage before people begin using an agent in practice.
Understanding demand, how people use agents in practice, and what level of Credits different services require is now an important part of our pilot work.
In some cases, colleagues encountered usage limit messages. This made it clear that Copilot Credits affect the day-to-day experience of using an agent, not simply how the service is licensed behind the scenes.
What this meant for our pilots
Many of the teams involved in our early pilots, including HR, Legal and Procurement, deal with frequent questions and requests for information and guidance. They are exactly the kinds of services where we expect AI agents to have the greatest impact.
They are also the kinds of services where usage could increase quickly once people start to see value in them.
As our pilots expanded, we found ourselves working through questions around licensing, Copilot Credits and governance alongside the technical development. In some cases, pilot activity paused while these issues were resolved.
What we’ve learned
Our experience creating internal AI agents has been overwhelmingly positive. We are seeing real potential for agents to help colleagues find information more quickly, navigate complex guidance and reduce repetitive work.
Early feedback from our pilots has been encouraging. Colleagues have told us they can find information more quickly, and in some cases tasks that previously involved searching through multiple pages and documents can now be completed much more efficiently.
At the same time, one of our clearest lessons is that building a useful agent is only part of the challenge.
As our internal pilots continue, we’re learning as much about licensing, access and adoption as we are about the technology itself. By sharing our experiences, we hope our members can spend less time navigating the mechanics and more time exploring where AI agents can genuinely make a difference.
Find out more by visiting our Artificial Intelligence page to explore publications and resources, learn more about our communities and sign up for our AI Literacy training.
For regular updates from the team sign up to our mailing list.
Get in touch with the team directly at AI@jisc.ac.uk