VM host capacity

What a reseller can read for capacity planning, how fresh it is, and how often to pull it

Implementing this with an agent or code generator? Give it vmhost-capacity.md (also at https://api.rackcorp.net/api/rest/vmhost-capacity.md). That file is the request bodies, the fields that do not mean what they look like, and the pull ceilings.

What you can see

Capacity planning should use the current snapshot. Each VM host reports CPU, memory, storage, traffic, and VM count about once a minute, and the API returns the latest row for every host in one call. That row is a point in time. Host-level free CPU, RAM, and disk are not stored as a history. If you need a trend, save the snapshots you pull.

Per-VM charts (CPU, memory, disk IO, guest agent) come from the RRD tables. Those queries aggregate a large sample store and are slow. Use them when someone opens a chart. Do not put them on a timer.

Prometheus. Scrape setup, metric names, and the duplicate-series check are on the Prometheus page. GET /devices/vmhost/prometheus is this fleet snapshot in text format. The path is /api/rest/v2.10/devices/vmhost/prometheus. Use the API UUID and secret as HTTP Basic. Set scrape_interval to 5m. The body is cached for 5 minutes per customer, and each device_id is emitted once.
scrape_configs:
  - job_name: devicevmhost
    scheme: https
    metrics_path: /api/rest/v2.10/devices/vmhost/prometheus
    scrape_interval: 5m
    scrape_timeout: 30s
    basic_auth:
      username: API_UUID
      password_file: /etc/prometheus/rackcorp_api_secret
    static_configs:
      - targets:
          - api.rackcorp.net

How fresh the data is, and how often to pull it

The API does not currently reject a client that polls too fast. Pulling faster than the table below does not return newer numbers, and RRD calls in particular add load on the stats database. Build the client to stay inside these ceilings.

Call What changes, and when Pull no more often than
device.vmhost.availability.getall Host check-in about every 60 seconds. outOfDate is true after 20 minutes without a check-in. Metric columns can lag the heartbeat by a short queue delay. 5 minutes for a fleet dashboard. Once a minute is the fastest the numbers can change. Do not poll every few seconds.
GET /devices/vmhost/prometheus The fleet snapshot above, as Prometheus gauges. Same rows, same freshness. 5 minutes. Faster scrapes are served from that cache and do not run the query again.
device.vmhost.get Host inventory (name, DC, status). Changes when a host is added or retired. 15 minutes, or when the page loads.
storage.vmhstorage.getall Pool byte counts, collected on the host every 5 minutes. stale is true when observedAt is older than 15 minutes. 10 minutes for the whole fleet. One call, not one call per host.
device.vmhost.vms.get Per-VM CPU share from a ~60 second hypervisor sample. Guest memory percentage from the guest agent. lastGuestMetricTs is the guest sample time. 5 minutes per host, and only for a host someone is looking at. Do not walk every host every minute.
monitoring.metrics.get Latest guest CPU, memory, and disk for one VM. Same guest sample as above. 5 minutes per VM. This is a point lookup, not a fleet sweep.
monitoring.data.get RRD history. New samples about once a minute, then rolled into 5-minute averages and 30-minute averages. See History. Not on a timer. One device and one series when a chart is opened, or a small nightly export. Prefer a window of 8 hours or longer.
A practical planner stores device.vmhost.availability.getall every 15 minutes in your own database. That gives you host free-space history without ever touching the RRD store. Fifteen minutes is also a good cadence for storage.vmhstorage.getall if you want byte-level pool trends.

computeCapabilities on the snapshot is a separate report from the host: light fields about every 5 minutes, heavier probes about every 15 minutes, and an unchanged payload is re-sent at least every 15 minutes. poolsStale inside that object means the pool sizes were taken from a storage inventory older than 15 minutes.

GET Fleet capacity snapshot

device.vmhost.availability.getall — one row per active VM host. REST: GET /api/rest/v2.10/devices/vmhoststatus. This is the call to build a capacity dashboard from.

Request

POST https://api.rackcorp.net/api/rest/v2.10/json.php
Authorization: Bearer YOUR_JWT_TOKEN
Content-Type: application/json

{
  "cmd": "device.vmhost.availability.getall"
}

Response

code, message, matches, and devices are on the top-level object. Each resource object has free, used, total, status (OK, WARNING, CRITICAL), and unit.

{
  "code": "OK",
  "message": "Successfully Retrieved Stats",
  "matches": 1,
  "devices": [
    {
      "deviceId": 12345,
      "deviceName": "vmh12",
      "dcId": 4,
      "dcName": "Sydney",
      "dcShort": "SYD1",
      "vmhVersion": "1.4.2",
      "outOfDate": false,
      "svcInfo": "N/A",
      "svcDescription": "DHCPOK NICSOK DISKOK",
      "svcCodes": ["DHCPOK", "NICSOK", "DISKOK"],
      "storage": { "free": 2400, "used": 18000, "total": 21000, "status": "OK", "unit": "GB", "raidStatus": true },
      "memory": {
        "free": 80, "used": 400, "total": 512, "status": "OK", "unit": "GB",
        "swapUsed": 2, "swapTotal": 64, "swapFree": 62,
        "oversubscribePct": 0.4, "combinedPct": 69.8
      },
      "cpu": { "free": 40, "used": 8, "total": 48, "status": "OK", "unit": "%" },
      "trafficInbound": { "free": 860, "used": 140, "total": 1000, "status": "OK", "unit": "Mbit/s" },
      "trafficOutbound": { "free": 910, "used": 90, "total": 1000, "status": "OK", "unit": "Mbit/s" },
      "vmCount": { "free": 55, "used": 25, "total": 80, "status": "OK", "unit": "VMs" }
    }
  ]
}

How to read the numbers

Field Meaning Status
cpu Logical CPUs, from the 5-minute load average, refreshed about once a minute. total is the CPU count. used is busy cores, free is idle cores. The unit string is %, but the values are core counts. Idle percent is free / total. CRITICAL below 20% idle, WARNING below 30% idle.
memory Gigabytes. used and total are what the host reported. free is headroom after projected allocations and a 32 GB reserve, floored at 0. When swap is known, oversubscribePct is swap used as a percent of real RAM, and combinedPct is RAM plus swap used over RAM plus swap. When swap is reported, status follows oversubscribe: OK at or under 10% of RAM, WARNING above 10%, CRITICAL above 40%. That replaces the RAM-headroom status.
storage Gigabytes of the host data volume. used and total are raw. free is the host's reported free space (already net of the host's own reserve) plus any projected allocation. If that sum is under 64 GB, free is returned as 0. raidStatus is the RAID health flag. For byte-accurate pool free space, use storage inventory. CRITICAL under 100 GB free, WARNING under 500 GB free.
trafficInbound / trafficOutbound Average Mbit/s on the public interface since the previous sample (about 60 seconds). total is the configured link speed, often 1000. CRITICAL under 100 Mbit/s free, WARNING under 200 Mbit/s free.
vmCount Running guests. total is a planning soft cap of 80, not a scheduler limit. WARNING at 70, CRITICAL at 80.
partition_root / partition_temp Free gigabytes on / and /home/temp, when the host reported them. total is a 30 GB display cap, not the real filesystem size. Use free. WARNING under 3 GB free, CRITICAL under 1 GB free.
outOfDate True when the host has not checked in for 20 minutes. Drop that row from planning until it clears.
svcCodes Health tokens, separate from capacity: DHCPOK / DHCPFAIL, NICSOK / NICOK / NIC FAIL, DISKOK / DISKWARN / DISKFAIL. NIC counts are also returned as network_nics_up, network_nics_down, network_nics_total, and network_status once the host has reported them. svcInfo is FAIL or WARN when one of those tokens is bad. Otherwise N/A.

GET Host list

device.vmhost.get — identity and location only, no utilisation. REST: GET /api/rest/v2.10/devices/vmhost. Optional customerid narrows the list to one customer. Optional groupby of country or dcname buckets the list.

{
  "cmd": "device.vmhost.get",
  "groupby": "dcname"
}

Each host includes DeviceID, DeviceName, DeviceDCID, DCCode, DCName, DCCountryCode, CountryName, DeviceStatus, and port speeds. Use DeviceID as vmhostDeviceId or deviceId on the calls below.

GET Storage pool bytes

storage.vmhstorage.getall is the byte-level view of volume groups on hosts that have reported storage inventory. The key needs storage access. The snapshot's storage.free is the right fleet headline; this call is the one to use when you are deciding whether a pool can take another volume.

{
  "cmd": "storage.vmhstorage.getall"
}

storage.vmhstorage.get with deviceId returns one host, including the raw inventory. On the list, each host has totalBytes, usedBytes, freeBytes, logicalVolumeCount, replicatedVolumeCount, observedAt (unix time), and stale. volumeGroups breaks the same totals out per group. storageCapabilities.ioEngines is present when the host reported an io-engine pool (capacity, used, free, committed bytes). Garbage-collection reclaimable bytes are garbageCollectionBytes.

The host collects this inventory every 5 minutes. Treat stale: true (older than 15 minutes) as missing data.

GET VMs on one host

device.vmhost.vms.get explains which guests are consuming a host. Call it when a planner opens a host. It is a heavier query than the fleet snapshot (guest metrics, contract vCPU, and mounted storage per VM).

{
  "cmd": "device.vmhost.vms.get",
  "vmhostDeviceId": 12345,
  "role": "ANY"
}

role is ANY (default), PRIMARY, or SECONDARY. The response has a vmhost object, matches, and vms.

Field Meaning
hostCpuSharePercent This VM's share of CPU activity among active VMs on the host. Active VMs sum to about 100. Taken from hypervisor jiffies when any VM has a rate, otherwise from guest or hypervisor CPU percent weighted by contracted vCPU. 0 when the VM is inactive, null when there is no sample.
memPct Guest RAM used, percent, from inside the VM. null if the guest agent has not reported.
memoryGbAllocated RAM allocated to the VM, from the device record. Use this when memPct is missing.
oversubscribePct Guest swap used as a percent of guest RAM.
storageTotalBytes Sum of mounted volume sizes, or the device storage figure when there are no mount rows.
lastGuestMetricTs Unix time of the latest guest memory sample. Ignore memPct if this is old.

GET Latest guest metrics

monitoring.metrics.get is the current guest sample for one VM (not the VM host). It reads the latest-value cache, so it is cheap compared with a history query, but it still should not be fanned out across every VM on a tight loop. The key needs monitoring access. deviceid is the VM.

{
  "cmd": "monitoring.metrics.get",
  "deviceid": 67890
}
  • cpu — percent. Prefers the hypervisor sample (H_CPU_USAGE_PCT), then the guest agent (G_CPU_USAGE_PCT). source says which. null if neither exists.
  • memory — guest RAM percent, plus ramUsedKb, ramTotalKb, swapUsedKb, swapTotalKb, oversubscribePct, and combinedPct when the agent sent them.
  • disk — one object per partition the agent has reported: usage_pct, size_kb, used_kb, avail_kb, mount_id.

Each block includes timestamp (unix). That timestamp is the freshness signal. Guest agents report on their own schedule, so do not assume a new sample every minute.

GET History

RRD reads are slow. monitoring.data.get runs an aggregate over the sample store for one device and one series. Short windows hit the 1-minute table, which is the largest. Do not poll it. Do not loop every VM and every series. A chart open, or a few series in a nightly job, is the intended use. Windows of 8 hours or more are cheaper because they read the rolled-up tables.

There is no caller-supplied interval. The API picks the table from the length of fromtime..totime (unix timestamps; default is the last hour).

Window Table Point spacing Kept for
Under 8 hours 1-minute samples 2 minutes about 12 hours
8 hours to 48 hours 5-minute averages 12 minutes 14 days
Longer than 48 hours 30-minute averages 30 minutes 3 months

Some series never use the 1-minute table, even for a short window. Guest CPU and guest memory (G_CPU, G_MEM) are read from the 5-minute table. Guest network socket counts and process counts (G_NET, G_PROCESSES) are read from the 30-minute table only. Asking for the last 15 minutes of those series does not give you 1-minute points.

Request

{
  "cmd": "monitoring.data.get",
  "deviceid": 67890,
  "snmpid": "VMCPUSTATS",
  "fromtime": 1731000000,
  "totime": 1731086400
}

Series that matter for capacity

snmpid Device What comes back
VMCPUSTATS VM CPUJIFFIES, a rate (jiffies per second) derived from the counter.
VMMEMSTATS VM ALLOCATED and RESIDENT, in MB. Gauges.
VMIOSTATS VM READ and WRITE, MB/s.
VMIOSTATSDEEP_<drive> VM Per-drive IO, busy time, and queue time. Heavier than VMIOSTATS. One drive per call.
G_CPU VM (guest agent) Cores, 1/5/15 load, and usage percent. Separate series per key. 5-minute table at finest.
G_MEM VM (guest agent) RAM and swap, KB and percent. 5-minute table at finest.
G_DISK VM (guest agent) Per-partition size, used, available, and percent. Only partitions the agent has registered.
PUBLICTRAFFIC Device Public in/out, Mbit/s. The outbound series is negative so a chart can draw it below the axis.

Points are [timestamp_ms, value] arrays, grouped by series name, plus snmpID. Rate series (CPU jiffies, traffic, disk IO) are a per-second delta. Gauge series (memory, guest percent) are the averaged level. A counter reset shows up as a gap, not a spike.

Anything outside the retention window is gone. You cannot ask for 1-minute CPU from last week; that table only holds about 12 hours. For host-level free space over weeks, store the fleet snapshot yourself.

Device checks and alerts

These are per-device checks, separate from the host capacity snapshot above. deviceid, alertid, and monitorid are lowercase. A monitor with interval 0 is disabled. monitoring.monitor.disable sets that interval to 0. monitoring.monitor.enable restores it, using 60 seconds when the stored interval is already 0.

CommandParameters
monitoring.alerts.getdeviceid. Optional resStart (default 0) and resWindow (default 100). Returns active alerts: alerts, recordsTotal. Each alert has id, monitorId, snmpId, name, description, severity, handled, created, lastAlert, duration.
monitoring.alert.silencealertid. Hides this firing until it triggers again.
monitoring.alert.silence-permanentalertid. Ignores the alert and sets the monitor to stay ignored.
monitoring.alert.handledalertid. Marks the alert handled and pauses the monitor so a flap does not reopen it.
monitoring.alert.handled-temporaryalertid. Same effect as monitoring.alert.handled.
monitoring.alert.clearalertid. Marks the alert cleared.
monitoring.alerts.silence-alldeviceid. Silences every active alert on the device. Returns affected.
monitoring.alerts.acknowledge-alldeviceid. Same effect as monitoring.alerts.silence-all.
monitoring.alerts.clear-handleddeviceid. Clears alerts that are already handled.
monitoring.monitor-types.getNo parameters. Returns types: id, name, mib, and configuration when the type has one.
monitoring.monitors.getdeviceid. Returns monitors.
monitoring.monitor.getmonitorid. Returns monitor, including ignoreSsl.
monitoring.monitor.createdeviceid, monitorTypeId. Optional interval (default 60, seconds), alertInterval, alertEnabled, alertEmail, tripped, trippedAutoReset, ignoreSsl, and data1 through data7 (the type's configuration says what those fields mean). Returns monitorId.
monitoring.monitor.updatemonitorid, plus any create field you want to change, including monitorTypeId. An empty alertEmail or dataN clears that value. interval 0 disables the check.
monitoring.monitor.deletemonitorid
monitoring.monitor.enablemonitorid
monitoring.monitor.disablemonitorid

A planning client that will not hurt the platform

  1. Every 15 minutes, call device.vmhost.availability.getall once and store the rows. Skip any device with outOfDate: true.
  2. On the same cadence, or every 10–15 minutes, call storage.vmhstorage.getall once if you need pool bytes. Skip stale hosts.
  3. When a person opens one host, call device.vmhost.vms.get for that host. Cache it for at least 5 minutes.
  4. When a person opens one VM's chart, call monitoring.data.get for that VM and that series. Cache the response for the rest of the session.
  5. Do not schedule monitoring.data.get. Do not call it once per VM in a loop.

Error responses

A failed call returns code: "FAULT":

{
  "code": "FAULT",
  "message": "Could not find device"
}
  • The device id is wrong.
  • vmhostDeviceId or deviceid was omitted.
  • monitoring.data.get was given an snmpid that has no graph.
  • The key is not allowed to call that command (device, monitoring, or storage permission).