I thought I’d share some very initial thoughts on Copilot for Microsoft 365, as I’ve had it for a few weeks.
It’s been fairly easy for people to try most of the AI tools. That’s not the case with Copilot for Microsoft 365, as it can only be purchased and enabled by central IT teams. This feels like it’s leading to a lot of AI FOMO (fear of missing out) – maybe that’s just me! That’s partly my reason for sharing early thoughts.
My headline view is that at the moment I’m a little disappointed, and usually end up going back to ChatGPT Plus, but it might well be that at the moment I haven’t quite figured out how to use it well. Or maybe my expectations were too high.
As I’ve mentioned before Microsoft have caused a lot of confusion with the name ‘Copilot’, using it for multiple products, so I’ll start by clearing that up a bit. For the purpose of this, there are two important versions:
- Copilot – the version that institutions get as part of most academic Microsoft licenses, which can access the web, and keeps and information you upload private to the institution, in the same way as say, Sharepoint.
- Copilot for Microsoft 365 – the version that costs $30 a month, and is built onto Microsoft 365 (ie Microsoft Office) plus it can also access anything else you have access to in your organisation’s 365 environment, so in Teams, Outlook etc.
In the web browser, if you have access to both of these, you can toggled between them, somewhat confusingly as as ‘Work’ and ‘Web’.
I have access to both of these, as well as the paid version of ChatGPT Plus, and the Google Gemini equivalent etc for comparison.
Let’s just go through a few examples.
Dealing with Emails
My verdict: very limited success
It’s fair to say, I’m not the best at managing emails. Like most of us, I get a lot, and, I think I can be honest and say that I have frequently been known to forget to reply, or remember about a week later that I’ve written a draft but not sent it, or even just overlook an email completely because at first glance I take it as spam or similar.
So, I thought Copilot for Microsoft 365 might well help.
I start by asking the version accessible through Teams:
Me: “What important emails have I not responded to?”
Copilot: “I’m sorry, but it seems there was an issue retrieving the specific emails you have not responded to. However, I can help you search through your emails or assist you with managing your inbox to identify important messages that require your attention. Would you like to proceed with that?”
I say ‘yes’ but it then tells me it’s going to look at mail, but doesn’t:
Copilot: “Certainly! I will search for important emails that you might have missed. Please give me a moment to gather the information for you.” (and then it does nothing – it doesn’t tell me anything more)
I wonder if the word ‘important’ is making it think of the ‘important’ flag in emails, so I try again:
Me: “what recent emails have i not responded to?”
Bingo! (Sort of…)
This time I do get a list of emails, but it’s a fairly random list, mostly irrelevant, include spammy sales emails, automated system emails etc. as I’ve not worked out a way of conveying ‘important’.
I give up on this. There’s potential, but at the moment it’s not really working.
Finding Information
My Verdict: Partial Success
I wanted to be reminded which institutions we are piloting Teachermatic with. This is documented in our files held in Microsoft Teams, and sounded like a good task for Copilot for Microsoft 365.
Me: “who are we piloting teachermatic with?”
It searches our Teams files, and does summarise the colleges, but missed the universities, which are in a separate document. The response is also somewhat verbose.
Copilot: “The TeacherMatic pilot is being conducted with several Further Education (FE) colleges. The pilot aimed to trial TeacherMatic across different subject areas at different levels to evaluate the quality and level of content generated by TeacherMatic. The FE colleges involved in the pilot include (…Lists the colleges here..) These institutions were chosen from over 60 applicants to participate in the pilot, which was divided into two phases: the first from May 2023 to June 2023, and the second from June 2023 to February 2024.”
Me: “Do you know which universities?”
It tells me it can’t find that information, and instead tells me some other fairly random facts about our pilots:
Copilot: “The search results do not specify universities involved in the TeacherMatic pilot. However, Jisc, the organization responsible for the pilot, typically runs three pilots a year with 5-15 participants each, which could include universities. The pilots involve identifying, piloting, and evaluating AI applications, with a typical post-pilot route via Jisc licensing team . For detailed information on specific universities involved, it may be necessary to consult Jisc’s pilot program documentation or contact the Jisc team directly.“
I remember the document probably says ‘Higher education’ rather than Universities so I try:
Me: ‘What about higher education institutions?
This time, curiously, it gets it partially correct, and then again tells me a lot of random extra information.
Copilot: “The TeacherMatic pilot in higher education is being explored with the University of Chester as one of the HE pilot institutions. The pilot aims to evaluate the generative AI platform TeacherMatic in higher education settings, focusing on its potential to assist educators and managers in creating resources, teaching materials, and administrative tasks 1. For more detailed information on other specific higher education institutions involved, it would be best to refer to Jisc’s pilot program documentation or contact the Jisc team directly.”
I decide to look for the document myself, and I find it’s got the title including the words ‘HE Institutions’. So one more try, even though I know the answer:
Me: “Who is in the HE Teachermatic pilot?”
Bingo again! I get the results.
Copilot: “The HE TeacherMatic pilot includes the following institutions:
-
- (…lists the institutions here)
These institutions are participating in the pilot to evaluate the generative AI platform TeacherMatic in higher education settings, focusing on its potential to assist educators and managers in creating resources, teaching materials, and administrative tasks”
What’s going in here though? One of the things I like about ChatGPT is I can be pretty loose with language. It has no problem if I use HE, Universities, Unis, Higher Education institutions etc. Copilot for Microsoft 365 though seemed very reliant on the actually file name to get to the info, A Microsoft 365 event seems to confirm this. So there’s potential here – if it had worked I’d find this sort of functionality really useful, but at the moment it feels very much like a work in progress.
Assisting with writing
My verdict: Limited success
Writing assistance is one of my main uses of ChatGPT Plus. I get it to proofread, suggest improvements, sense check and so on, quite often many times a day. I was really looking forward to doing this with in Word. It turns out though, that it doesn’t really do what I want.
It just gives the single option – ‘rewrite this’ which offers a new version:
This isn’t how I use ChatGPT.
I say things like
- ‘give me feedback on this. Don’t change anything, just let know how it could be improved’
- ‘give me feedback on this. Don’t change anything, just tell me about major issues’.
I then read ChatGPT’s suggestions, sometime take them onboard, sometimes ignore them, some times ask it to go ahead an make them. I never want it to just rewrite it.
I then tried to do something similar in the chatbox, rather than through the main user interface, by selecting texting and using the side bar to get it to focus. This seemed logical enough but it didn’t work.
I did, however, have more success telling it to give feedback by telling it which paragraph I wanted it to look at.
Me: “Suggest improvements for the highlighted text”
Copilot “Could you please clarify which text you are referring to?”
Me: “the second paragraph”
Copilot: “Here are some suggestions …. (goes on to give some reasonable suggestions)
Not a scalable option for long documents, so I’d love to hear other people’s approaches:
Maybe I’m being unfair on Copilot for Microsoft 365 here. If it was the only tool I had access to, maybe I’d think it was great. But for me, I want a guide and coach, not an automation tool. Small changes to the user interface would achieve this, so I wouldn’t be surprised to see improvements in this in the future.
Email chain summarisation
Verdict: I love this!
And now one that works really well for me. We all know the pain of this one – you are away for a couple of days, and come back to a massive email chain, with 20 or 30 messages. In Outlook I can now just click ‘Summarise’ and I’ll get a short and (so far!) accurate summary of the entire email chain, that also gives me a clickable link to exactly which email the point was make (for example an action agreed).
I can’t give an screenshot of this without sharing personal information (my email!), but it was pretty much spot on.
Summarising long documents
Verdict: Does the job well
If I’ve got a long document open, being able to summarise the document is useful. It’s not a game changer though, and in reality most documents I get that I want to summarise are PDFs from the web, so standard Copilot is more useful. Your use case might vary.
Creating PowerPoints
Verdict: Some serious shortcomings
If you’ve been to one of my presentations you’ll almost certainly have seen me show Gamma – one of my favourite tools. It creates presentations, and does so in a very fun and engaging way. We can now also create presentations in Powerpoint. To test this I use an example I also use in Gamma:
Me: “Create a presentation about Generative AI in Education: The Myths, the Threats, and the Opportunities“
Copilot: “Okay, here you go. A presentation about “Generative AI in Education: The Myths, the Threats, and the Opportunities” has been created with multiple slides. If you’d like, Designer also has other layout options to consider.”
This is what I get:
My eyes are immediately drawn to the colour scheme, and the lack of contrast. I run the PowerPoint accessibility checker, but it assures me:
“No accessibility issues found. People with disabilities should not have difficult reading this presentation”
This seems unlikely – I can barely read the slides myself. I verify this with an online contrast checker, and my suspicion is confirmed, with a terrible contrast ratio of 1.25:1
We always stress you must assess the output of AI and not just take it at face value, but this seems like such as basic thing to get wrong, that it’s hard to overlook.
For the sake of fairness, I create a second example, which is a little better. You can download my two examples and my Gamma creation here:
I think for me the Gamma presentation wins hands down – I’d happy give that presentation with a couple of minor tweaks, but the Copilot for Office 365 ones are barely a decent starting point, although they do give some short speaker notes.
And what else?
I’ve not touched upon the Excel features, partly because I don’t use Excel much myself, and partly because it’s still marked as ‘Preview’, and I’ve not mentioned the Teams features like meeting summarisation, as they are also part of Teams Premium, so not a Copilot for 365 specific feature.
Final thoughts
This wasn’t intended to be a thorough review of Copilot for 365. I just wanted to share my journey so far. I’ve not been that positive about it, and that might well change as it improves and I learn how to use it more.
I thought I was massively missing out not having access to it though. I was suffering big time from AI FOMO. I think there’s a lot of concern within Universities and Colleges that it’s a real problem that we can’t afford Copilot for 365 for all our staff and students. So institutional AI FOMO too maybe.
At the moment though, I think that we can do a huge amount with Copilot itself, the version that is already part of most of our Universities and Colleges Microsoft licenses, and which provides the all important corporate data security features. So maybe at the moment we don’t need to be too concerned.
I still end up going back to ChatGPT Plus most of the time. That’s $20 a month, going up to $30 a month for the Team version, so not hugely cheaper than Copilot 365.
If I didn’t have access to ChatGPT Plus, I’d be pretty happy with Copilot (the free version), but even if there was a way to do it, I wouldn’t pay the $30 a month for Copilot for 365 myself, unlike for ChatGPT Plus, which I happily pay for.
So at the moment, (and this is very much a personal view), we don’t need to worry about not being able to buy Copilot for Microsoft 365 for all our staff and students, but we should take advantage of Copilot (the basic version) and make that available, at least to all over 18s. As more people share their experiences, I’m sure we will find use cases where careful and specific use of Copilot for Microsoft 365 is well worth it.
And I reserve the right to change my mind if it improves, more use case emerge, or I get much better at using it.
And a call to action…
We’re aware that a number of Universities and Colleges are also trying Copilot for Microsoft 365, and we’d love to share experiences, even if it’s just that you are evaluating it but don’t have any firm outcomes. If you are and can share details drop us a email at ai@jisc.ac.uk and we’ll collate what is happening across the sector and share back in a future blog post.
Find out more by visiting our Artificial Intelligence page to view publications and resources, join us for events and discover what AI has to offer through our range of interactive online demos.
For regular updates from the team sign up to our mailing list.
Get in touch with the team directly at AI@jisc.ac.uk