logo-unlock-security

Tampering the RAM: a new way to break into any VM

Tampering the RAM: a new way to break into any VM

Despite the spread of containers, many organizations still rely on Virtual Machines (VMs). Many choose to distribute preconfigured images containing their web applications, to protect their source codes from prying eyes.

Some rely on strong login credentials, others grant access only via SSH, with proper certificates. Some disable the shortcuts of GRUB to obtain shells with root privileges, others hide the GRUB and the login, and start the web services only. Some may even encrypt the disk to avoid having it mounted externally.

When all these security measures are implemented simultaneously, you are left with no exposed attack surface but the web application itself. Should it be removed too, the VM could truly be considered unbreakable... couldn't it?

Page Cache Theory

Before explaining the attack technique, we must say a few words about "Page Caching" in Linux, being the principle on which the technique is based.

The Page Cache is a part of the Virtual File System (VFS) that aims to improve the I/O performance in Read/Write operations.

We say "page caching" because the Linux kernel works with memory units called "pages". So, instead of tracking and managing every single bit of information accessed by the system, the Page Cache manages the whole page containing that information. Pages usually have a size of 4 KB. Since they are the fundamental unit of the Page Cache, every request to read even one single bit will result in 4 KB of data being read.

The following simplified diagram shows the fundamental operations carried out by the Page Cache.

Page caching workflow upon read/write request

As shown in the previous picture, a read operation is based on three fundamental moments:

  1. A user-space application requests the kernel to read data from the disk using system calls such as read(), pread(), vread(), mmap(), etc.
  2. The Linux kernel checks whether the pages corresponding to the requested data are already in the cache and, if so (Cache HIT), it returns them straight to the caller. In this case, there is no access to the disk.
  3. If the requested pages are not in the cache yet (Cache MISS), the kernel saves enough space in the cache to store them. Next, it performs data reading from the disk. The data are then saved in the cache and returned to the caller.

From that moment on, any future request to access the file will result in a Cache HIT, so the disk will not be involved anymore.

Let's see again what happens upon a file reading request.

Detail of the file caching workflow in read operations

In the I/O operations that use the read() syscall, the kernel saves the pages corresponding to the requested data both in the Page Cache and in another application cache. With the mmap() syscall, on the other hand, this duplication does not take place and the kernel relies on the Page Cache only.

So, what do we expect to find in a vmem file?

Depending on whether we accessed the files with a read() or mmap() syscall, we expect to find one or more copies of the pages containing the data requested by all the system processes, plus any neighboring pages that the kernel may have cached for us to limit the access to the disk. Apart from this, we will also find all the data structures used by the kernel, the running processes, any unused memory regions, etc.

Page Caching, in practice

To give a practical demonstration of how page caching works, we will need:

  • sync – a tool that forces the writing of “dirty” cache pages to disk
  • /proc/sys/vm/drop_caches – a file on procfs that clears the cache.
  • vmtouch – a tool for the Page Cache diagnostics.

Setting up the environment

We create a test file that will be used to test the cache functioning during read operations.

$ dd if=/dev/random of=file count=128 bs=1M

We clear all the caches to start from a clean situation.

$ sync; echo 3 | sudo tee /proc/sys/vm/drop_caches

We can make sure that the test file has been cleared from the cache using the tool VMTouch and checking that the number of "Resident Pages" equals zero.

$ vmtouch file
           Files: 1
     Directories: 0
  Resident Pages: 0/32768  0/128M  0%
         Elapsed: 0.002936 seconds

Reading a file — read() syscall

We create a simple C program that reads 2 bytes from the test file.

#include <fcntl.h>
#include <unistd.h>
#include <stdint.h>

int main(int argc, char *argv[])
{
  int fd = open("file", O_RDONLY);
  uint8_t buf[2] = {0};
  read(fd, buf, sizeof(buf));
  close(fd);

  return 0;
}

We compile and run the program. Since the fundamental unit is the page, we expect the kernel to cache not only the two bytes requested, but the whole page containing them.

$ vmtouch file 
           Files: 1
     Directories: 0
  Resident Pages: 12/32768  48K/128M  0.0366%
         Elapsed: 0.00183 seconds

Unexpectedly, we don’t find just one page (4 KB) in the cache, but 12 (48 KB). That is because the kernel implements the so-called "read-ahead logic" — to reduce the disk accesses, it caches data that were not explicitly requested, but are likely to be accessed in the near future.

This behavior can be controlled using the posix_fadvise() syscall. Let's see how.

#include <fcntl.h>
#include <unistd.h>
#include <stdint.h>

int main(int argc, char *argv[])
{
  int fd = open("file", O_RDONLY);
  posix_fadvise(fd, 0, 0, POSIX_FADV_RANDOM);
  uint8_t buf[2] = {0};
  read(fd, buf, sizeof(buf));
  close(fd);

  return 0;
}

We clear the cache, compile and run the program. Then we check again the cache status.

$ vmtouch file 
           Files: 1
     Directories: 0
  Resident Pages: 1/32768  4K/128M  0.00305%
         Elapsed: 0.001783 seconds

Reading a file — mmap() syscall

We can achieve the same result using the mmap() syscall as well.


#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdint.h>
#include <stdio.h>

int main(int argc, char *argv[])
{
  int fd = open("file", O_RDONLY);
  uint8_t *buf = mmap(NULL, 2, PROT_READ, MAP_SHARED, fd, 0);
  printf("0x%02x\n", buf[0]);
  munmap(buf, 2);
  close(fd);

  return 0;
}

However, once running the program, we realize that the “read-ahead logic” of mmap() is even more aggressive and reads 32 pages upon a tiny request of 2 bytes.

$ vmtouch file 
           Files: 1
     Directories: 0
  Resident Pages: 32/32768  128K/128M  0.0977%
         Elapsed: 0.001935 seconds

Similarly to the previous case, we can control the behavior of mmap() using the madvise() syscall.


#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdint.h>
#include <stdio.h>

int main(int argc, char *argv[])
{
  int fd = open("file", O_RDONLY);
  uint8_t *buf = mmap(NULL, 2, PROT_READ, MAP_SHARED, fd, 0);
  madvise(buf, 2, MADV_RANDOM);
  printf("0x%02x\n", buf[0]);
  munmap(buf, 2);
  close(fd);

  return 0;
}

As we can see, now it cached just one page.

$ vmtouch file 
           Files: 1
     Directories: 0
  Resident Pages: 1/32768  4K/128M  0.00305%
         Elapsed: 0.001792 seconds

The paused state of a VM

When we use a virtualization system, we can pause the execution of the VM. Unlike suspending, that saves the VM state to disk, pausing it means having it saved on RAM.

Pausing is useful when it comes to temporarily interrupting the execution of an operation on the VM to resume it later.

The virtualization files

When we use a VM, some files are created in the host system. Among them, the main ones are:

  • vmdk – virtual disk corresponding to the VM disk
  • vmsd – metadata and other information concerning the VM snapshots
  • vmx – setup information and VM hardware settings
  • vmsn – VM status, related to a particular snapshot
  • vmem – memory contents when the VM is paused

The vmem file sparks our interest because it allows us to analyze the contents of the RAM of a paused VM, that will not change as long as the VM is paused.

Tampering the RAM

If we combine what we have said so far regarding the VM paused state and page caching in read operations, we get to the subject of this article, our new attack technique.

The attack is based on 3 steps:

  1. We start a VM and interact with it to force file reading
  2. We pause the VM and edit the Page Cache contents from the vmem file
  3. We resume the VM from its paused state and force again file reading

Let's now apply these 3 steps to a real case to understand how they actually work. We will use a VM downloaded from HackMyVM, though we could have used any other VM.

We want to obtain a root shell and we will do it in 3 steps:

  1. Username enumeration, forcing the reading of the /etc/passwd file.
  2. Password hashes enumeration, forcing the reading of the /etc/shadow file.
  3. Login bypass and privilege escalation, by tampering the RAM.

Username enumeration

When we start the VM, all that we have is the login screen.

Login screen on a Linux VM

Our first aim is to enumerate all the users on the system. In Linux, we know that the list of users, along with other user properties, is stored in the /etc/passwd file — that we cannot access, obviously.

What we do know is that the /usr/bin/login binary is in front of us. To keep it simple, we could say that this is a program indirectly executed by the init process via getty or a display manager. When executed, it prompts the user to state the username and password to login with. Before checking that the password is correct, the program checks that the username exists in the /etc/passwd file.

Knowing this, we just have to pause the VM as soon as we see the login screen and open the related vmem file with an hex editor. At this point, we look for matches of the string :x:1000:1000:: — so, for the string of the /etc/passwd file that refers to the user with ID 1000 (which is usually given to the first users generated on Linux).

Username enumeration done by researching the VMEM file of a paused VM

In this way, we are able to read the contents of the /etc/passwd file and get the full list of the system users, including the user moksha:


moksha:x:1000:1000:moksha,,,:/home/moksha:/bin/bash

Password hashes enumeration

Let’s now suppose that we want to login with the user moksha and look for a match for the string moksha:$ in the vmem file — so, for a line in the /etc/shadow that begins with the password hash of the user moksha.

In this moment, we would get no results. This is because the /etc/shadow file can be accessed only upon login attempt.

So, we should login with the user moksha and a random password to get the /etc/passwd and /etc/shadow files read — and, subsequently, have the related pages of both files saved to the Page Cache memory.

So, let’s attempt a login as specified and pause the VM.

Login with correct username and wrong password

Let’s now open again the vmem file corresponding to the VM RAM and search the pattern moksha:$ once more.

/etc/shadow file cached in the VM memory thanks to file caching

We do have a match this time, so we can read the contents of the file /etc/shadow and get the password hash of all the users on the system, including moksha.

moksha:$y$j9T$aahAz65DkZxl.its6aXmV1$7beXMp4sTYWPV9IuR1CIbPqidBBFx7zhqq1nt2IBTl2:19361:0:99999:7:::

Login bypass and privilege escalation

Once we got the usernames and the password hashes, we could try to get the credentials in plain text and use them to log in. Or we could overwrite the hashes with a hash for which we know the password, leveraging the Page Cache once again.

We generate the hash for the password "password", specifying a salt of the same length as the one used in the current hash. The result will be a hash of the same length.

>>> import crypt
>>> crypt.crypt(
...   'password', # Our password of choice
...   '$y$j9T$aahAz65DkZxl.its6aXmV1$' # The salt found in the vmem file
... )
'$y$j9T$aahAz65DkZxl.its6aXmV1$oHbJ46tHyqV/QPv8c5lxYh.vHfIC1ovf7GBWpu1tIT1'
Replacing the password hash in the VMEM file of the VM

At this point, we could go one step further: by modifying the user ID (UID) and the group ID (GID) from 1000 (i.e. user moksha) to 0 (i.e. root user), we could gain access with the highest privileges.

We can do it by searching for all the occurrences of moksha:x:1000:1000: and replacing them with moksha:x:0000:0000:, making sure to use a 4-zero sequence for both the UID and GID, to get the same string length.

Now we can save the vmem file and unpause the VM to log in with the credentials moksha:password and gain access to the VM with root privileges.

Privilege escalation by tampering the RAM of a VM

Conclusions

This technique does not exploit any vulnerabilities, but leverages the operational mechanisms of operating systems and virtual machines. Therefore, there is currently no way to make this technique harmless, except for completely disabling the Page Cache — which would, however, lead to severe slowdowns in the VM.

Some virtualization systems, like VirtualBox, try to avoid RAM tampering by adding at the beginning of the VMEM file a checksum calculated on the file contents, preventing the RAM from being restored if tampered. It is clear, however, that in that case it would be sufficient to recalculate the hash or change the virtualization system to regain access to the VM as explained.

Francesco Marano
Francesco Marano
CEO | Cyber Security Consultant
www.unlock-security.it

Amo far fare ai software cose diverse da quelle per cui sono stati progettati!Ciao, sono Francesco e sono un esperto di cyber security con anni di esperienza come Penetration Tester. Nel tempo libero svolgo ricerche in ambito sicurezza per trovare nuove vulnerabilità. Sono speaker ad eventi di settore per parlare delle mie ricerche.Oggi sono alla guida di Unlock Security, un'azienda specializzata in offensive security.

Related Posts