hugepages for non-default pagesize need manual setup

Where do 1G hugepages get exposed by default? or don't they?

OK tried this and they don't get exposes afaict.

This feels like this generally needs improving, with automatic mounting of hugepage FS for different pagesize options - so for example:

   /dev/hugepages1G
   /def/hugepages

with appropriate apparmor changes as well for libvirt to allow access to common paths on Ubuntu systems.

@jjo

Use of 'default_hugepagesz=1G' might also tweak things a bit.

FWIW I think kernel boot command line options should be the preferred way of allocating hugepages as its generally the most reliable way of doing it.

On Thu, Feb 2, 2017 at 12:58 PM, James Page <email address hidden> wrote:

> This feels like this generally needs improving, with automatic mounting
> of hugepage FS for different pagesize options - so for example:
>
> /dev/hugepages1G
> /def/hugepages
>
> with appropriate apparmor changes as well for libvirt to allow access to
> common paths on Ubuntu systems.
>

I agree,
I faced the same when we faced the need for a basic hugepage allocation
service for DPDK and wanted to share a few of my thoughts back then.
BTW by starting early the dpdk service so far had no problems, but when
increasing and restarting later on that was an issue sometimes.
Yet there the kernel option would not help without reboot either.
It has a few fallbacks like dropping caches when trying to get more huge
pages to mitigate this.
But it already says in the config comments that it is a basic helper and
admins need to cover more complex cases.

FYI:
Config:
https://gerrit.fd.io/r/gitweb?p=deb_dpdk.git;a=blob;f=debian/dpdk.conf;h=a5aea865a8c43623761f9299549cbd2b25fd06a9;hb=refs/heads/16.11.x
Code:
https://gerrit.fd.io/r/gitweb?p=deb_dpdk.git;a=blob;f=debian/dpdk-init;h=103488edce6592a5f93e1b32cd5d476416374f81;hb=refs/heads/16.11.x

Back then I noted a todo that at some point all applications will need a
joint hugepage allocation strategy - just what you suggested here.

In addition to your correct statement of some (configurable) default paths
there also should be a way to work together in a better way.
One program sets 2x1G pages, the next 4x1G and the next 200x2M - what gets
to the boot parm?
Also since these don't know of each other they often interfer with each
other.

I can't find my old notes, but at the core in addition to what you suggest
it was about establishing a common shared way to be reused and not stomping
on each other.
In my vision it was like:
/etc/hugepaged/config <- core config
/etc/hugepaged/config.d/ <- every Package/Admin could drop their need
(libvirt, dpdk, databases, java, users adding custom things)

The service would then process the aggregate of that - ensure that it has
no conflicting specifications and allocates the sum of all those configs.
It could auto generate kernel commandline args to be added to make
allocation more reliable and still fall back to late allocation as we do
today.

Note: there is also a package hugepages (2.19-0ubuntu1) which provides:
  This package contains a number of utilities that will help administrate
the
  use of huge pages on your system. hugeedit modifies binaries to set
default
  segment remapping behavior. hugectl sets environment variables for using
huge
  pages and then execs the target program. hugeadm gives easy access to
huge page
  pool size control. pagesize lists page sizes available on the machine.

Back then I thought creating a proper service infrastructure as addon to
that project might be a good way to go.
Of course the systemd task suggests that this might also be a way to go.

E.g. also /etc/default/qemu-kvm also adds an extra mount to /run/hugepages/kvm if you set KVM_HUGEPAGES=1

So yeah a unified approach is what we should strive for these days.

Status changed to 'Confirmed' because the bug affects multiple users.

please reopen if this is still an issue

Yes, having a centralized default available multi-size hugepage mountpoints would still be nice to have.