Thursday, June 6, 2024

00000101

How time flies! I have now been 5 years at Microsoft. I started as a Program Manager, which changed into Product Manager, and now I’m an Engineer.

image

Despite the title changes the work I do has not changed much over these five years. I have always been a jack of many trades, and I enjoying doing many things at once. When everything is interesting and challenging, it is hard to cut something out. And I have been lucky to be allowed to work to my strengths while still learning – and I love to learn!

What has changed over the years is that I have learned to appreciate process and governance more than before. Which is why I have onboarded to internal roles as privacy champ, compliance champ and responsible AI champ – the last one becoming increasingly important as Microsoft focus all energy on Copilot. Luckily for me loving search as a function and search tech, is that Microsoft Search plays an important role also in Copilot.

What the next one or five years will bring I have no idea, and it’s not important. I’m still a day to day kind of person, with no long term plans. If what I do is fun, challenging and benefits Microsoft customers, I will keep doing it. And for those who know me, I’m not giving up om my long term mission to cut ties from classic SharePoint search features still having dependencies in the modern world of Microsoft Search. Some cuts are hard but still important to ensure the success of future experiences.

Cake time!

Wednesday, March 27, 2024

Copilot Productivity Tip – Structured formatting in Word

Time spent per day before: 15 minutes per document
Time spent with Copilot: 5 minutes per document

Resources



Taking my previous post about re-writing text in Word, we can prompt to have key points from the text pulled out and as an example formatted as a table for easier consumption.

image

The prompt used is “Extract key points into a table with two columns for key point header and details”, and the output is the table below.

Key point  

Details 

Copilot can rewrite the content 

Copilot can check and improve the tense, the passive voice, the readability, and the personal tone of the content. 

Copilot may or may not respect the writer's style 

Copilot's suggestions may vary depending on the instructions given by the writer. The writer can try to specify the style they want and see what Copilot does. 

Friday, March 22, 2024

Copilot Productivity Tip – Word re-write

Time spent per day before: 30-60 minutes per document
Time spent with Copilot: 5-10 minutes per document

Resources




Writing in a language that is not one's own can be challenging. That is why Copilot can help rewrite the content to make it more consistent and clear. Copilot can correct the tense, the passive voice, the readability, and the personal tone of the content. This way readers can enjoy the content more and understand the main points better.

Some writers may have a specific style that they want to use. Copilot in Word may or may not respect that style, depending on the instructions given. But one can always try to tell Copilot what kind of style they want and see what happens. Maybe Copilot will surprise them with some creative suggestions. Or maybe Copilot will ignore them and do whatever it wants. Who knows?

Original draft

Here’s the original blog-post I started out with before throwing Copilot at it.

Not everyone is a native writer of the language they produce content in. Thus, using Copilot to rewrite your content for consistency in terms of tense, passive language, readability, and personal flavor will ensure those consuming the content have a better experience and more clearly pick up on the points you want to come across in your argumentation.

If you have a particular writing style, then Copilot in Word may or may not be able to assist, but you can try to add this as part of the prompting instructions and verify the outcome.

Automatic rewrite

When using the auto rewrite feature this is what I ended up with.

image

Sometimes, people write content in a language that is not their native one. Therefore, using Copilot to edit your content for consistency in grammar, voice, clarity, and tone will help those who read the content have a better experience and understand the messages you want to convey in your reasoning.

If you prefer a specific writing style, then Copilot in Word may or may not help you, but you can attempt to include this in the prompting instructions and check the result.

Manual prompt for rewrite

This time around I wanted to give more instructions to simplify my writing.

“Rewrite the text using shorter sentences and language understood by an audience with little experience in Copilot or Word. Make instructions concise and relatable for beginners.“

Do you write in a language that is not your first one? Then Copilot can help you make your writing better. It can fix grammar mistakes, make sentences easier to read, and use the right words for your message.

Do you have a favorite way of writing? Copilot in Word might be able to copy it. Just tell it what you want and see what it does. You can always change it if you don't like it.

Manual prompt with styling

And this is the prompt I ended up with for the final post, as it reflects some of my personal writing style. Maybe this will makes it way into Word eventually using writing samples for style.

“Rewrite the text using shorter sentences and language understood by an audience with little experience in Copilot or Word. Make the text longer than the original, write in third-person tense and **DO NOT** use *I*, *You*, *Your* and *We*. Throw in some subtle irony and sarcasm.”

Wednesday, March 20, 2024

Copilot Productivity Tip – Teams Message Rewrite

Time spent per day before: 10 minutes, rethinking the words to use
Time spent with Copilot: 1-2 minutes, letting Copilot do the work

Resources


One of the challenges when writing is to convey feelings or emotion on what you type. I like to be short and concise in my messaging, but sometimes this may not be the best way to communicate when talking to people you don't know, or even people you know.

Let’s take the message “I don’t agree with your points” and run it thru Copilot to make it longer and Enthusiastic.

image

The output can be something like: “Thank you for sharing your perspective. However, I have a different view on the matter and I would like to explain why I disagree with some of the points you made.” which is less likely to piss Vesa off :)

image

You can even do a customized prompt for formatting: “Make it sound like it was written by Super Mario”. The above prompt is actually a prompt hack to overcome Responsible AI to add some Paolo Italian flavor into the message.

image

Monday, March 18, 2024

Copilot Productivity Tip – Teams Chat Catchup

Time spent per day before: 30-60 minutes
Time spent with Copilot: 5-15 minutes

Resources


My workday at Microsoft involves communication with multiple teams spread around the globe, which means in-person communication is not always possible. This is why Teams has become the go-to tool for many conversations and discussions. When I start my day at 8am in the morning I know chats has happened in other time zones, and instead of reading everything right away I can use Copilot catchup for a summary, and then decide if I need to read it all.



I typically use the “Summarize what I've missed” Copilot suggested prompt inline in the chat which opens the Copilot pane, or I open the pane manually as seen below with prompts such as “Highlights from the past day” or “Highlights from the past 7 days”.

image image

Friday, March 15, 2024

Copilot Productivity Tip – Teams Meeting Insights


Time spent per day before: 0-120 minutes
Time spent with Copilot: 0-15 minutes

Resources

The beauty of online meetings with transcripts is the ability to quickly go back and find key points later without having to watch the recording or read thru the full transcript. 

By default Teams provides an AI notes section with a quick summary, and using the Copilot pane you can ask more direct questions such as summarizing your talking points, or ask what your action items were for the meeting.

My best example is a late night meeting where I forgot to take notes and the next morning I knew I was supposed to contact an “Andrew”, but had forgotten the full name. Requesting the name in the meetings Copilot quickly gave me the name, saving me around 15 minutes and some grief.




Friday, March 8, 2024

Allowing arbitrary custom scripting in SharePoint Online, or not? – that is the question! (aka Stealing your data since 2001!)

…and the answer is, as it always has been, NO!



Disclaimer: the opinions of this post are mine entirely, and nothing to do with my work at Microsoft. I have not changed opinion on this matter in the past many many years.

Sparked by the recent message center post MC714186 – Remove Custom Script setting in OneDrive and SharePoint web, I figured I’d write out my stance and my full support of the planned change.

Summary: The Custom Script setting in OneDrive and SharePoint web will be removed in March 2024. A new PowerShell command, "DelayDenyAddAndCustomizePagesEnforcement", has been introduced to delay the change to custom script set on the Tenant until mid-November 2024. The NoScriptSite setting will be configured to True for all existing SharePoint sites and OneDrive sites except for specific site templates. Existing scripts in OneDrive and SharePoint sites will remain unaffected. Administrators can permit the execution of custom scripts on specific SharePoint sites using the Set-SPOSite command.

In the above summary I want to point out that “existing scripts will remain unaffected” means classic page injections. Not SPFx solutions on with the setting requiresCustomScript=true.

The challenge

SharePoint by design is a JavaScript application and has always allowed for extensibility in the UX. In modern ways extensibility is achieved via the SharePoint Framework (SPFx), which you should use, and that is ok. Because by virtue of how SPFx is built any script or logic in a SPFx solution is governed and controlled by design.

To explain what I mean. By arbitrary code I mean someone can insert and freely modify JavaScript that runs in a SharePoint site or page without anyone noticing. One example being a bad actor put script on a page which when a person visit the page then transfers all the persons OneDrive files to evil.corp in the background without the person ever knowing. And this has been possible with SharePoint in classic experiences since always, and also with 3p modern webparts when allowing script to run, or by bypassing settings for a site.

When writing code in a SPFx solution, it is no longer arbitrary. Someone wrote it, built it, and then handed the package over to a person with SharePoint admin role rights to install it (or by an admin role proxy if a site has it’s own app catalog). How diligent the process is for code review and installation is up to each organization, but at least the setup allows for a governed process to help control what is installed into a tenants SharePoint Online experience.

Arbitrary script on the other hand would be JavaScript added to SharePoint sites or pages by any regular user with edit rights or a site administrator. Every Team or Group creator is a site administrator of the associated site, and this is not very controlled in my opinion. With classic SharePoint, script can be injected or allowed via the Script Editor web part, via search display templates, and via something called custom actions. When Modern sites were introduced many years ago, allowing these features were by default turned by setting the “Allow and Customize Pages” permission to false. The feature is more commonly renamed in e.g. PnP PowerShell to NoScriptSite. Documentation on turning this on or off can be found at https://learn.microsoft.com/en-us/sharepoint/allow-or-prevent-custom-script.

Back to the message center post and what changes

  • In March, the SharePoint admin UX setting to allow custom script on OneDrive sites or self-service sites is going away, and is being replaced by a PowerShell cmdlet function Set-SPOTenant DelayDenyAddAndCustomizePagesEnforcement which you can use until mid-November 2024.
  • All new and existing sites will be set to block custom scripts unless they were created with any of these web templates: BLANKINTERNETCONTAINER#0, CMSPUBLISHING#0, BLANKINTERNET#0. GROUP#0, APPCATALOG#0, CSPCONTAINER#0.
  • From mid-November 2024, custom script is disabled for your sites every 24 hours. You can manually re-enable it with for example Set-SPOSite <SiteURL> –DenyAddAndCustomizePages. This means you need to set up a scheduled job every day to re-enable it where needed for SPFx solutions relying on this feature, unless you migrate your ungoverned script solutions to governed ones using SPFx.

Should you be worried?

If you are still on classic pages, then disregard as you are a lost cause anyways. It’s 2024 people! Modern is no longer modern. It is the new classic.

Hopefully most customers have moved over to using SharePoint Framework for any custom JavaScript solution running in SharePoint Online by now. However, the elephant in the room is my infamous Modern Script Editor Web Part- https://github.com/pnp/sp-dev-fx-webparts/blob/main/samples/react-script-editor/README.md. If built without changes then someone has changed the site settings to allow arbitrary script to run for the web part to work. When this is then flipped off every 24 hours by the service, the web part will stop running (which is good!).

Instead of working the web part will render ugly white space instead.

image
Alternatively you rebuild the web part bypassing the custom script check saying it does not rely on custom script (boooooo on you, and bad governance practice).

It is no secret what my take is on usage of this web part. The sample modern script web part should NEVER be used, and I will argue to my death that there are no good scenarios for it. Sure, it’s quicker to use compared to always creating new SPFx extensions or web parts, but is this ease of use more important than security? My answer is still a big NO! Sure, you may reduce the attack vector by using site scoped app catalogs, and maybe you have the utter most control for who can edit pages in those sites. But is this a long term risk you are willing to take?

Why not just build your scenario solutions in a governed way thus blocking someone from changing the code? If you have access to someone building the Modern Script Editor web part, then you also have access to someone who can do it the right way.

What now?

My hope is that people do not run a script every 24h to allow custom script, or that they cheat on building SPFx web parts to bypass scripting. Instead everyone should embrace this opportunity to get their solutions cleaned up – if they haven’t already.

Happy coding governed solution, and preventing your files from being stolen!

Thursday, January 18, 2024

Easier editing of Microsoft Search verticals in SharePoint sites (finally fixed!)

This is perhaps a tiny one, but for the longest time search verticals in a SharePoint site or SharePoint hub site has been a bit of a mystery when you wanted to edit a vertical. To invoke edit, you had to double click the vertical, as there was no Edit button in the ribbon.

The issue has been known since the feature rolled out, but not addressed. I'll be as blunt to say I decided to take matters into my own hands and just fix it. So here you go, a gift from me to allow a more intuitive way of editing search verticals in SharePoint - as the Microsoft 365 Admin center has had the Edit button always.

At last the experience matches the documentation at https://learn.microsoft.com/en-us/microsoftsearch/manage-verticals#manage-site-level-verticals.



Thursday, January 11, 2024

Demystifying Author properties on files in SharePoint and search

demystify

For as long as I have been doing SharePoint, figuring out what properties to use when filtering or displaying search results in regards to people has been a challenge as the documentation in this space is somewhat lacking. I’m not sure why I haven’t done this writeup earlier, but no time like the present.

Friday, December 8, 2023

AI model bias and why responsible technology matters – exemplified by image generation

In this era of AI where ChatGPT with LLM’s has become the hottest topic in computer science since the Apple Macintosh and the IBM PC, I figured I’d do a small write-up on AI model bias and why paying attention to bias is important. This is especially true when it comes to enterprise scenarios where Microsoft is launching a long range of AI powered copilot experiences.

The author of this article works for Microsoft (December 2023) and is an internal champion for responsible ai and technology as well as internal champion for privacy and compliance.

At Microsoft we have a high bar for delivering responsible AI solutions, which means there is a lot of work put in place to ensure the output from AI systems follow Microsoft’s AI principles to be fair, inclusive, reliable and safe, privacy and security is accounted for, and the systems are accountable.

Any model, that be a large language model (LLM) or a model to generate image will inherently have bias built in due to the training data used. In smaller models you can manually verify the training data to counter some bias and balance the training set, but as models grow large this becomes inherently harder. I’m not saying there are not systems in place to counter training bias already, but to truly counter bias this has to be built into pre and post processing of input prompts and outputs from the models.

I will use image generation as an example where I will show the difference between using image creator in Microsoft Designer (https://designer.microsoft.com/) built on DALL·E 3 from OpenAI and Stable Diffusion XL, which is an open source model from Stability.AI (https://stability.ai/). The Microsoft solution has guardrails in place, where the open source solution do not – unless you add them yourself via prompting. Not saying neither of them are perfect as examples will show.

I want to call out that any bias shown is not statistically verified, and only based on generating a set of sample random images with the same prompt.

Example 1 - photo of correctional officer in a well lit hallway eating a donut

image

The above eight images are from DALL·E 3. They are all close-up photos showing a fit, light skinned male with dark hair.

image

In comparison the SDXL images have a wider focal point showing the full body. It’s a mix of male and female people, and also a mix of light and dark skinned people. I would argue the SDXL model is more accurate to what people look like in 2023, while the DALL·E 3 model output “perfect” looking people. If this is due to the images the models are trained on, or the prompt being augmenting to have “perfect” looking people I do not know.

The default color palette is also different where DALL·E 3 has more green and SDXL has more brownish colors.

If I add “overweight” to the DALL·E 3 prompt, the Responsible AI filter kicks in and blocks the generation. If I add “fat”, then it works.

image

With SDXL I can modify the prompt to closeup photo of a slim white male correctional officer in a well lit hallway eating a donut” to mimic what DALL·E 3 output by default – to counter the wide angle and real life looking people bias of the model.

image

Example 2 – woman

Let’s try a simple prompt with the subject “woman”. For SDXL I added negative prompting to avoid any nsfw images – which is blocked as part of DALL·E 3 RAI principles.

image

DALL·E 3 seems to pivot towards portrait photo’s when no extra contextual information is given, as that is likely the intent with a simple input subject. They are also all dark haired and seem to be young women.

image

In comparison SDXL gives a wide variety of image types, pivoting to more art-like images instead of photos.

Example 3 – painting of a beautiful norwegian fjord with vikings, with a boing 737 in the sky, in the style of munch’s scream

image

The DALL·E 3 painting nails the airplane and pretty much the painting style of Edward Munch.

image

The SDXL one is not bad either, but the Munch style is not as visible for this one sample. And the scale of the plane vs. the viking ship and buildings is way off.

Learnings

These simple examples shows that articulating your intent in prompting is crucial. Either the system has to add guardrails and contextual information to the prompt, or the person prompting has to be articulate on what they want returned and what they do not want returned. And you have to generate many images to find that ONE you really like.

For online services like Microsoft Designer going the safe route is the only approach as people using the service comes from a wide variety of backgrounds and age groups. Taking that extra measure to ensure everyone feels safe is important to trusting the service.

Open source solutions you can run on your own PC/phone/table can allow for less guardrails as the individual running it likely has more skill and is using the tool themselves. Maybe the analogy of hiring a carpenter as a service vs. hammering yourself can be used. You trust a hired professional to meet a certain bar, while you are responsible yourself on anything you do.

When it comes LLM’s we know they are largely based on English text today, and would favor input and output in this language. As they are built on public data, that will influence default writing style as well. fortunately ChatGPT and Microsoft Copilots put a lot of effort into the system prompts put around the user prompt, to counter any bias in the model. This is to ensure grounding in facts and avoid hallucinations. More on that for another post.

References

I used the service of https://designer.microsoft.com/image-creator to create the DALL·E 3 images, and I used the Draw Things app on a MacBook with an 8-bit quantized version of the default SDXL model. The Draw Things app also work on iOS devices.