These days the development of software is a rapidly changing process. The introduction of cloud hosting and containerization has made big applications running in your server room a thing of the past.

Where before you would maintain one or two big projects with a team of engineers dedicated to a single project. Now you have hundreds of small services split into bite-sized chunks with developers working on several projects.

In the perfect world, every software project will have a full suite of unit, integration and system tests in place. Even in these rare cases, it is still nearly impossible to avoid an escalation or critical performance issue when you are running at scale in production.

In my role as a technical support engineer, I have seen all manner of escalations. 

They can come in the form of;

•  A performance bug in the product
• A corner case issue you wouldn't consider a user hitting
• A service being offline that the product depends on
• A third party library the product depends on being upgraded
• Many more!

From a support engineer's perspective, here are a few tips I have learned over the years on how to approach an escalation.

Let the customer know you have seen their ticket

The initial customer contact in support is crucial. Letting the customer know their issue is under investigation reassures the customer and takes the pressure off the support engineer.

The initial response from support can often be automated, these days using a bot for first-line support is often standard practice.

A simple example of an email would be:

 "Dear {{ticket.requester.name}},

Thank you for reaching out to the {{team.name}}, a ticket has been created for you with the Ticket ID - {{ticket.id}}.

A support engineer will get back to you as soon as possible and work with you to resolve your issue.

Thank you for your patience.

Sincerely,

{{engineer.name}}"

This simple email lets the customer know their issue is assigned to an engineer and the team is willing to help. You could enhance this email by adding elements such as:

• Common issues / solutions section
• A personal sign off specific to one engineer
• A phone number or chat to use if the situation is critical
• A link to a support portal so they can see their ticket status

If the support case comes in via phone or chat, this is not so much of an issue. The customer is getting live responses from the support engineer.

Get the information you need

Investigating an issue can be simple, unfortunately, this is often not the case. If the product you support is complicated, escalations could be caused by a wide variety of factors.

The goal of the support engineer is to find the route cause of the issue quickly, with as little time to set up a reproduction case as possible. 

When the customer sees an issue, getting their logs, screenshots and environment data will often show the root cause of the problem. You can get this information through email or chat, but a call is usually the quickest way to access this information.

Having a template email or call script to get this information in place will ensure a new support engineer can gather all the essential data. They can use this data to consult with the other engineers if they need assistance.

With this information, an engineer will be able to report the issue without needing to reproduce the issue themselves, drastically reducing the time taken to resolve the issue.

If possible, convert the issue into an email

When you have all the information, you may need from the customer; It is often best to halt direct communication when you are investigating the issue.

If you are on a call or chat with the customer, filtering through log files and attempting to reproduce the issue can be a demanding process. Often it is better to convert the call or chat with the customer into an email. The customer will typically understand if you tell them you will get back to them when you know more about the issue.

Having to only focus on finding the issue removes any distractions. Even if finding the issue is your sole task, it may take hours or in extreme cases days to find the exact cause.

Be clear on the issue priority

So now you have your issue, this process could have taken you five minutes or five days. At this point, the development team must be notified.

How the team is notified will change based on the process within the engineering team. Often the two conventional approaches are to raise an alert in a chat room or create an engineering ticket.

For a severe/blocking issue such as a service being unavailable, raising an alert in the office chat will alert the engineers, and they can restart the services as appropriate. It may be they already know there is an issue and are investigating.

For an issue in the product, like missing data or a page in the UI erroring; A ticket to fix the product must be created. The priority of an issue is often not something you as a support engineer can control, but you can advise.

The priority is based on several factors, such as:

• How many customers it affects
• What it prevents a user from doing
• If the issue can be worked around
• Who reported the escalation

For example;

• If the issue has been reported into support five times in the last 24 hours, this is something hitting a lot of customers. Not fixing the issue right away will cause the number of reports for the problem to build up in support.
• If the issue prevents the user from logging in, or from seeing any data in the software, this has a higher priority than if it prevents them from clicking a link to the documentation.
• If a user has to uninstall your product to continue their task, this has a higher impact. If they have to refresh every 6 hours to get the UI working, it should be fixed, but it can wait.
• If the issue is reported by a customer who generates 60% of the revenue for your company, it must be treated as a higher priority than a customer who is on a free trial.

You should consider these factors when creating a ticket and your rationale for your suggested priority should be documented on the ticket. This will give the engineers some context for when they will start work on the issue.

Keep the customer in the loop

As soon as the issue has been reported, it becomes something you as a support engineer has no control over. Until the developers resolve the issue, all you can do is keep the customer updated on changes.

It is essential to let the customer know the issue has been reported. If you have a ticket ID or public issue board providing them with the link to this means they can check the status themselves whenever they need.

Promising hard deadlines on when the issue will be resolved can be risky. If an issue is more complicated than it initially appears the fix may be delayed which could potentially annoy the customer. Being vague on a fix date is often the best solution, using "as soon as possible", "in the next release" or "we hope to fix the issue within the week" are good alternatives to utilize.

Having a specific inbox or state for a ticket waiting on the development team gives you an easy 

If it has been a while since the ticket was created, it is often beneficial to let the customer know they have not been forgotten.

When the developers have resolved the issue, you can finally notify the customer and resolve the ticket. This entire process could take less than an hour, or it could take months. However long it takes you to get the satisfying feeling of making the customer happy.

‍