Dec 15th 2021
Grafana, the open-source analytics and visualisation platform that powers monitoring dashboards, has this week disclosed a critical path traversal vulnerability. The bug (tracked as CVE-2021-43798) has received a CVSS score of 7.5 and is classed as high severity.
The bug affects Grafana versions 8.0.0 to 8.3.0 and allows attackers to navigate outside of the Grafana folder and access normally restricted files on the server, including passwords and configuration files.
The path traversal vulnerability was found within the plugins directory and as all Grafana installs include plugins by default, every hosted Grafana 8 installation was vulnerable. Grafana Cloud was not vulnerable to this vulnerability. With exploits taking place in the wild, Grafana took the decision to release a blog post outlining the timeline of this vulnerability.
Grafana was notified of the vulnerability on December 3rd by a security researcher. This issue was fixed in a matter of hours and committed to Git. Grafana had then planned a release timeline, with private customers and large cloud providers being provided with the patched release on December 7th and December 14th suggested for the public release.
However, this plan started to unravel quickly. On December 6th, Grafana received a second report of the vulnerability and on December 7th they were notified that the vulnerability was public. By this time, proof of concept code was being shared across Twitter. In light of this, Grafana rapidly evaluated its release timeline and decided to go public with the exploit, releasing both the private release and public release on December 7th.
At the time of writing, a search on Shodan.io shows a total of 87,335 public Grafana instances worldwide, with researchers estimating that 3,000-5,000 were currently vulnerable. This stresses the importance of patching.
In some good news, it was found that Grafana instances located behind a reverse proxy that normalises the PATH of the request will mitigate the vulnerability. So, if patching is not possible, that might offer some temporary reprieve.
For those looking to detect attempts to exploit this vulnerability, an experimental Sigma ruleset can be found on GitHub.
For those interested in the technical aspects of this vulnerability, security researcher Tom Hudson (@TomNomNom) posted details of the relevant code which detailed how easy this mistake was and the results of ignoring warnings.
The developers had attempted to sanitise the user-supplied file path parameter using the inbuilt function Clean() to prevent exactly the scenario encountered.
requestedFile := filepath.Clean(web.Params(c.Req)[“*”])
pluginFilePath :=filepath.Join(plugin.PluginDir, requestedFile)
On initial reading, this looks acceptable, as the Clean() function replaces “../” with “/”. It seems that the function has a slight quirk that drives home the importance of reading documentation.
Eliminating “..” elements that begin a rooted path “:”, replace "/.." by "/" at the beginning of a path.
The key part of this is that it will only eliminate elements where the path begins with “/” and any leading sequences with “../” do not get removed. As these path traversal characters were not removed, they then get passed to the Join() function which adds them to the plugin file path that gets interpreted.
Overall, this seems an extremely easy mistake to make, and we have to express sympathy for the Grafana developer who is no doubt having a bad week. There was, however, a warning presented to the developer by the GoSec code scanning software to warn of this vulnerability.
The code had been commented to say that this generated an error but that the path was already clean so the warning could be ignored. The security alert was then commented out so that it did not alert again. In hindsight, that was most definitely a mistake.
There are some important takeaways from this Grafana issue: