Skip to main content

The article outlines steps needed for extending storage space on FreeBSD.

title: Shuffling Partitions on FreeBSD
author: Michael Lucas
date: September 23, 2014

I've recently moved my personal web sites to [Vultr], using virtual machines
instead of real hardware. I didn't track server utilization, so I provisioned
the machines based on a vague gut feeling.

The web server started spewing signal 11's, occasionally taking down the site by
killing mysql. Investigation showed that this server didn't have enough memory.
How can 1GB RAM not be enough for WordPress and MySQL? Why, back in _my_ day--


Right. Sorry about that.

Anyway, I needed to increase the amount of memory. This meant moving up to a
larger hosting package, which also expanded my hard drive space. After running
`gpart recover` to move the backup GPT table to the end of the new disk, my new
disk was partitioned like so:

    $ gpart show

    =>        34  1342177213  vtbd0  GPT            (640G)
              34         128      1  freebsd-boot   (64K)
             162           6         - free -       (3.0K)
             168   666988544      2  freebsd-ufs    (318G)
       666988712     4096000      3  freebsd-swap   (2.0G)
       671084712   671092535         - free -       (320G)

I have 320 GB of space space at the end of the disk.

The easy thing to do would be to create a new partition in that space. I
advocate and recommend partitioning servers. The only reason that this system
has one large partition is because that's what the hosting provider gave me.

I'm writing a book on FreeBSD disk partitioning, however, so this struck me as
an opportunity to try something that I need for the book. (As I write this, you
can still get [FreeBSD Mastery: Storage Essentials][1] at a pre-pub discount.)
How would I expand the root partition, with the swap space smack dab in the
middle of the disk?

Virtualized systems have no awareness of the underlying disk. Advice like "put
the swap partition at the beginning of the disk" becomes irrelevant, as you have
no idea where that physically is. On a system like this, how would I use the
built-in FreeBSD tools to create a swap partition at the end of the disk, and
expand the existing partition to fill the remaining space?

This isn't as easy as you might think. FreeBSD's [gpart][2] command has no
feature to add a partition at a specific offset. But it can be done.

Any time you touch disk format or partitions, you might lose filesystems. Back
up your vital files. For a WordPress web server, this is my web directory and
the SQL database. (My backup includes a half-complete version of this article.
If my repartitioning goes badly, I'll re-title this piece "How to Not
Repartition." But anyway...) Copy these files off the target server.

Now, what exactly do I want to do?

* I want 4-5 GB of swap space, at the end of the disk (the server now has 2GB RAM).
* I want to remove the current swap space.
* I want to expand the root partition to fill the remaining space.

`gpart(8)` won't let me say "create a 4GB partition at the end of the disk." It
will let me create a filler partition that I have no intention of actually
using, however. As I'm sure the disk is not precisely 320GB, I'm going to play
it safe and give myself 5GB of room for this swap partition. I give this
partition a label to remind me of its purpose and role.

    $ gpart add -t freebsd -s 315GB -l garbage vtbd0
    vtbd0s4 added

The partitioning now looks like this.

    =>        34  1342177213  vtbd0  GPT            (640G)
              34         128      1  freebsd-boot   (64K)
             162           6         - free -       (3.0K)
             168   666988544      2  freebsd-ufs    (318G)
       666988712     4096000      3  freebsd-swap   (2.0G)
       671084712   660602880      4  freebsd        (315G)
      1331687592    10489655         - free -       (5.0G)

Now I can add a swap partition at the end of the disk.

    $ gpart add -t freebsd-swap -l swap0 vtbd0
    vtbd0p5 added

The resulting partitioning looks like this.

    $ gpart show

    =>        34  1342177213  vtbd0  GPT            (640G)
              34         128      1  freebsd-boot   (64K)
             162           6         - free -       (3.0K)
             168   666988544      2  freebsd-ufs    (318G)
       666988712     4096000      3  freebsd-swap   (2.0G)
       671084712   660602880      4  freebsd        (315G)
      1331687592    10489655      5  freebsd-swap   (5.0G)

Tell `/etc/fstab` about the new swap partition, and remove the old one.

    /dev/gpt/swap0 none swap sw 0 0

> In looking at the old entry, I realized that [Vultr] uses `glabel(8)` labels,
> where I use `gpart(8)` labels. Either type is fine, but I need to remember
> that `/dev/label/swap0` is the old swap partition, and `/dev/gpt/swap0` is the
> new one.

Activate the new swap space. I could reboot, but why bother?

    $ swapon /dev/gpt/swap0

My swap now looks like this.

    $ swapinfo -h

    Device            1K-blocks  Used  Avail  Capacity
    /dev/label/swap0  2047996    0B    2.0G   0%
    /dev/gpt/swap0    5244824    0B    5.0G   0%
    Total             7292820    0B    7.0G   0%

Turn off the old swap.

    $ swapoff /dev/label/swap0

The old swap is unused. I can put it out of my misery. Double-check gpart show
to learn which partition is your swap space (3) and your temporary placeholder
(4). Double double-check these numbers. We're going to remove these partitions.
If you delete your data partition due to your own stupidity you will be...

    $ gpart delete -i 3 vtbd0
    vtbd0p3 deleted

    $ gpart delete -i 4 vtbd0
    vtbd0s4 deleted

**Triple double-check:** do you still have a root filesystem? (Yes, FreeBSD has safeguards to prevent you from deleting mounted partitions. Check anyway.)

    $ gpart show

    =>        34  1342177213  vtbd0  GPT            (640G)
              34         128      1  freebsd-boot   (64K)
             162           6         - free -       (3.0K)
             168   666988544      2  freebsd-ufs    (318G)
       666988712   664698880         - free -       (317G)
      1331687592    10489655      5  freebsd-swap   (5.0G)

Our swap space is at the end of the disk. And we have 317GB of free space right
next to our root filesystem. You have not ruined your day. Yet.

**Double-check your backups.** Do you really have everything you need to
recreate this server? If so, expand the root filesystem with gpart resize. Don't
specify a size, and the new partition will fill all available contiguous space.

    $ gpart resize -i 2 vtbd0
    vtbd0p2 resized

    $ gpart show
    =>        34  1342177213  vtbd0  GPT            (640G)
              34         128      1  freebsd-boot   (64K)
             162           6         - free -       (3.0K)
             168  1331687424      2  freebsd-ufs    (635G)
      1331687592    10489655      5  freebsd-swap   (5.0G)

Now I have a 318GB filesystem on a 636GB partition. Let's expand that filesystem
to fill the partition. You can't resize a filesystem label such as
`/dev/label/root0`, you must use a partition identifier like `vtbd0p2` or
`/dev/gpt/rootfs0`. In FreeBSD 10, you can use `growfs` on mounted file systems.

    $ growfs /dev/vtbd0p2

    It's strongly recommended to make a backup before growing the file system.
    OK to grow filesystem on /dev/vtbd0p2 from 318GB to 635GB? [Yes/No] yes
    growfs: /dev/vtbd0p2: Operation not permitted

Not permitted? I activated GEOM debugging mode by setting **kern.geom.debugflags**
to **0x10**, but was still denied. I've grown mounted filesystems before, so what
the heck?

This virtual server has disabled soft updates, journaling, and all the fancy
FreeBSD disk performance features. I suspect this error is tied to that. Let's
go to single user mode and grow the filesystem unmounted.

I reboot, and get:

> `Mounting from ufs:/dev/label/rootfs0 failed with error 19.`

Even when you know what's wrong, this message makes that little voice in the
back of my skull simultaneously call me an idiot and scream "You destroyed your
filesystem! Ha ha!" Plus, I can no longer make notes in my web browser--the
article is on the non-running server.

Fortunately, I know which partition is the root partition. I enter

    $ mountroot> ufs:/dev/vtbd0p2

and get the familiar single-user-mode prompt. Now I can do:

    $ growfs vtbd0p2

I answer `yes`, and new super-blocks scroll across the screen. The filesystem
grows to fill all available contiguous space.

My suspicion is that resizing the partition destroyed the label. Many GEOM
classes store information in the last sector of the partition. Let's use a GPT
label instead.

    $ gpart modify -i 2 -l rootfs vtbd0

Mount the root filesystem read-write.

    $ mount -o rw /dev/vtbd0p2 /

Create a new `/etc/fstab` entry for the root filesystem, using the GPT label
instead of the `glabel(8)` one.

    /dev/gpt/rootfs / ufs rw,noatime 1 1

And then a reboot to see if everything comes back. It does.

My partitions now look like this:

    $ df -h

    Filesystem         Size    Used   Avail Capacity  Mounted on
    /dev/gpt/rootfs    615G    6.1G    560G     1%    /
    devfs              1.0K    1.0K      0B   100%    /dev
    devfs              1.0K    1.0K      0B   100%    /var/named/dev

All installed disk space is now in use. Mission accomplished!

Having written this, though, I have no chance of forgetting that I need to go
back and do a custom install to partition the server properly.