Looked into this a bit more. All tubing and every filter will create some amount of flow restriction, and therefore some pressure drop from atmospheric when and engine tries to pull air through it. This device is simply measuring the pressure drop from atmospheric to a point downstream of the filter. Any further restrictions downstream of wherever the device is connected will be invisible to it. Where was the device connected to the intake?
It appears it is calibrated to somewhere around 20" H2O. Keep in mind however that when you measure intake vacuum you are looking at inches of Hg. 20" H2O is less than 1.5" Hg, so this is a pretty sensitive measurement.
When does the filter restriction affect hp? Well that depends - anything that reduces the volume of air drawn in will reduce max hp, so in theory any restriction limits max hp. However with some carbs it will be much worse. If the carb has a bowl vent direct to atmosphere any filter restriction causes the mixture to get richer. If the bowl vents to downstream of the filter (like in any car from later years of carbs), then the mixture is more constant - but still not as constant as a modern EFI system. Not sure about dirt bike carbs, but any all-position carb is especially sensitive to intake restriction as they don't provide a mixture that is constant with airflow anyway.
That said it must be seeing some pressure drop greater than usual, but I'm not convinced how important that is, or how unusual it is on a vehicle. Especially at other than max hp. The ECU could probably detect this based on air flow, throttle position and rpm, but I don't think any bother with it.