Synwin

• Introduction
• Files
• Requirements
  • Compatibility and Contraindications
• Usage
  • Installation
  • Operation
  • Notes
• Theory of Operation
  • Synthetic Disk
  • Preamble
  • Network Bridge
  • Cleanup
• Other Aspects
  • Hardware Spoofing
  • Machine Identities
• Credits
• Legal

The goal here is to enable "dual booting" the same Windows installation, on both the real hardware, and, in a virtual machine under Linux.

One installs Windows once, on the real hardware, alongside Linux, in the traditional dual-boot scenario. After power on of the physical host, one chooses which OS to boot, Linux or Windows. Booting Windows works the same way it always has. Booting Linux works the same way it always has. However, after choosing to boot Linux, one can then boot the Windows instance inside a VM, running under Linux.

One only needs to install, configure, update, and maintain the single Windows instance. Boot Windows on the real hardware when appropriate for the circumstances, and run Windows in a VM when that will do. Circumstances in which one might want to boot Windows on the hardware include performance demands, hardware compatibility, and/or testing.

synwin-1.0.tar.gz — Distribution archive, containing the below

Within the distribution archive are:

Temporary Files

These are created by synwin_prep, and removed by synwin_done:

Either of the above may deleted if found after QEMU has exited.

• Microsoft Windows, already installed on the bare metal
• Just about any reasonably-recent, general-purpose Linux distribution (also installed on the bare metal)
• Linux packages/features (Debian package names given in [square-brackets])
  • Linux kernel with KVM (Kernel Virtual Machine) features [linux-image]
  • QEMU virtual machine manager [qemu]
  • Open VM Firmware / TianoCore firmware image [ovfm]
  • gdisk partitioning tool [gdisk]
• UEFI boot (not BIOS nor CSM)
• Partitioning
  • GPT-based (not MBR/BIOS)
  • A single Windows partition
  • Microsoft-recommended partition order (ESP, MSR, Windows)
  • Windows on the same device as the ESP
• A working knowledge of the Linux/Unix operating environment. This is not a tool for the Linux novice, sorry.

I tested this under Debian 12 (bookworm) running the pre-compiled 6.1.0 kernel, and QEMU 7.2.11. I had previously tested it under Debian 11 (bullseye) with QEMU 5.2.0. Anything of vaguely similar capability should work.

The partitioning requirements are mostly because the shell scripts that set all this up are not designed to handle variations or complexity. They'll just abort when they hit anything non-trivial. Feel free to improve upon them.

• The virtual machine environment will almost certainly not be compatible with Secure Boot nor TPM usage. If Windows is set to require either of these, Windows will likely fail to boot in the VM. Windows 10 does not require either, in a default config.
• BitLocker using TPM keys will fail the same way. BitLocker using a PIN or key-on-USB should be workable, although I have not tried it.
• Disabling "Fast Startup" in Windows is strongly recommended. (In fact, I recommend this even if you are not using Synwin.)

The good news is, anything that doesn't work in the VM, should still work when Windows is booted on the bare metal.

1. Copy the Bash scripts to a directory in your PATH (such as /usr/local/bin/).
2. Copy synwin.conf to an appropriate location. The scripts check for a config file in the following locations, in order:
   1. In the file pointed to by the SYNWINCONF environment variable
   2. In the current working directory (./synwin.conf)
   3. $HOME/.config/synwin.conf (but see note below)
   4. /etc/synwin.conf
3. Edit synwin.conf in the text editor of your choice, and customize appropriately for your installation. The file is well-commented. Follow the instructions therein. Do not skip this step.

Note that the scripts must be run as two different users (root and your regular user). This makes keeping a config file relative to one's home directory more complicated, since the value of $HOME will vary.

1. Boot Linux
2. Run synwin_prep as root
3. Run synwin_run as your regular user. Windows will boot and run.
4. Use Windows as you like. Switch back to Linux as you like.
5. Shutdown Windows as you normally would
6. Run synwin_done as root

Normal operation for synwin_prep is for it to report the details of the syndisk (device mapper map) it created.

If DEBUG is defined in the environment, the scripts will emit debugging output to standard error. Knowledge of the source code will generally be needed to interpret it.

The first time you boot the Windows instance in the VM under Linux, it will go through a hardware detection cycle, and likely require a reboot — perhaps more than one. It is Windows, after all.

You may have to manually install device drivers for the virtual hardware.

You may have to pre-install the Windows drivers for the VirtIO display adapter with Windows running on the bare metal (that is, before you try to boot Windows in the VM). The VirtIO virtual adapter may not provide sufficient VGA fallback capability, preventing Windows from booting unless the drivers are already ready.

When running in the VM, in Device Manager, you may see "Microsoft Basic Display Adapter" with a yellow-bang failure status, alongside the working virtio device. This seems to be benign.

When using the QEMU vc UI, I find I cannot ungrab input (return keyboard and mouse control back to Linux) unless the menu bar is visible. When using the stdio UI, I find I cannot ungrab input when in full-screen mode.

The synwin_prep script sets things up. It maps the synthetic disk, creates the network tap, and so on. It sets permissions such that synwin_run does not need to be run as root, but synwin_prep itself needs to be run as root to do so.

The synwin_run script invokes QEMU, with KVM acceleration (kernel virtual machine), using what was set-up in synwin_prep. The entire script is basically just building a giant QEMU command line. It is intended to be run as a regular user.

synwin_done does the opposite of synwin_prep. It also needs to be run as root.

"Run as root" can mean by login, su(1), sudo(8), or whatever.

The synwin_prep script sets up a device mapper node to present a "synthetic disk" representing the partitions a Microsoft Windows system needs to boot. The device mapper node will be a block device which appears identical to the real disk in all aspects important to Windows.

It is this "synthetic disk" or "syndisk" technique that lead to this package being called "synwin".

The syndisk will map in the Windows partition, the MSR partition (Microsoft Reserved), and the EFI System Partition (ESP). These are the real partitions; changes made in the VM will be written back to the real disk (unless you arrange otherwise with your VM software).

If you have any of these partitions mounted in your Linux system, you will need to dismount them before setting up the syndisk.

No other partitions are presented or copied; instead, blank disk areas are mapped in to fill the gaps. Reads will return all zeros. Anything written to these areas will be discarded. Presumably you don't want Windows messing with your Linux partitions anyway.

The start of the disk contains the MBR (Master Boot Record) and the GPT (GUID Partition Table). The MBR contains a boot block and the legacy partition table. Together I term this area (MBR+GPT) the "preamble". (This "preamble" is approximately the same as "the partition tables" or "disklabel", but technically the GPT includes a second copy at the end of the disk. The preamble is only the start of the disk.)

We can't map the actual preamble in using device mapper, because the Linux kernel holds the partition tables open when the filesystems within them are mounted. (At least, I assume that's what's going on. If I try, I get EBUSY.) So we copy the preamble out to an image file (one can read the preamble, just not map it), and map the image file into the syndisk instead.

The VM firmware and Windows will find its partitions, and their contents, the same way it does on the real hardware. (Anything looking closely at the partition tables will consider the GPT backup copy at the end of the disk to be invalid, as that part of the device map will return zeros when read. Fortunately, this generally does not matter, since it is just a backup copy. Anything that actually does care, should not be run against the synthetic disk.)

After use, the image file is discarded, along with any changes made by the VM to the preamble. If software is doing anything serious to the partition table from inside the VM with the synthetic disk, things are going to end badly anyway.

Typically one wants to be able to reach networks (local and/or the Internet) from the Windows VM. To make this possible for the common case, we bridge the Linux host's network interface to Windows using a network tap device. Network taps are a subset of the tun (tunnel) devices used in Linux for virtual network devices of all sorts.

This behavior can be disabled in the config file. See also the section on Machine Identities, below.

The synwin_done script attempts to undo what synwin_prep did. It removes the device mapper node, detaches the loop device, and deletes the image file with the copy of the partition tables. It also tries to mount the list of filesystems that given to dismount during prep (but in reverse order — that which is unmounted first is mounted last).

If you do not run the cleanup script, the Linux host may encounter trouble during shutdown, as the device mapper map will still be holding the loop device open, which will in turn prevent the filesystem containing the preamble image file from being dismounted.

Windows uses various hardware attributes to generate a machine ID used for license activation/tracking. If these attributes change, Windows gets upset and demands it be reactivated. To work around this, we can set values in the VM to be the same as those from the real hardware, to fool Windows into being happy.

Exactly which attributes matter are not well documented. (Microsoft presumably considers the user to be the enemy here, so documentation would be counter-productive from that stance.) However, I found that spoofing the make, model and serial number of the PC, mainboard, and hard disk, seemed to work well. All of this information can be obtained by running the dmidecode(8) utility under Linux, and then filling it in synwin.conf.

In a purely physical-machine world, a dual-boot system with Linux and Windows might reasonably assign the same machine identity (hostname and network addresses) to both. After all, only one can be running at a time, and they are the same machine.

With the capability of booting the same Windows installation on the real hardware or in a virtual machine running under Linux, the above strategy breaks down. If the Windows system comes up under Linux, one would now have two machines with the same name and address on the same network. That is not going to work.

My strategy is to assign the Windows installation a separate identity from the Linux installation. They have distinct hostnames, distinct IP addresses, and distinct MAC addresses. In effect, they function as if they were two completely independent machines on the network — which is, after all, the goal.

I generate a LAA MAC to use with Windows. I assign that to the virtual NIC used inside Linux. When Windows is booted on the real hardware, I find the physical NIC in device manager, and set the same LAA for that network adapter. Thus, regardless of how Windows is booted, it presents the same MAC address to the rest of the LAN.

I set up my LAN DHCP server to look for that MAC address and assign a static IP address (DHCP reservation), different from the one my Linux installation gets. (This part is optional; it will work just as well with dynamically assigned addresses, or manually-configured static IP addresses.)

Once this is done, I can even have the Windows instance in the VM use local network protocols to talk to the Linux host it's running under. I can access Samba shares on the Linux host from the Windows VM, or RDP from Linux to the Windows VM.

"If I have seen further, it is by standing on the shoulders of giants." (Isaac Newton)

Some of the ideas used here were gleaned from various online discussions on sites like Reddit and the Stack Exchange family. Unfortunately I neglected to note those origins, and such inspirations are now lost to the mists of time.

Ultimately this is just a half-baked front-end for powerful tools like QEMU and the Linux kernel. My work is minuscule by comparison.

This is free and unencumbered software released into the public domain.

Anyone is free to copy, modify, publish, use, compile, sell, or distribute this software, either in source code form or as a compiled binary, for any purpose, commercial or non-commercial, and by any means.

In jurisdictions that recognize copyright laws, the author or authors of this software dedicate any and all copyright interest in the software to the public domain. We make this dedication for the benefit of the public at large and to the detriment of our heirs and successors. We intend this dedication to be an overt act of relinquishment in perpetuity of all present and future rights to this software under copyright law.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

For more information, please refer to http://unlicense.org/