Does "Not In" Filter exclude empty values?

Yes, using Not In Filter always excludes (filters out) all the empty (null) values.

This configuration, inherited from the standard SQL practice, determines that: when comparing an Empty value to any other value, it is neither equal nor different, but indeterminate.
Therefore, empty values are never included when using an "In" or "Not In", because it's never going to be "equal" or "distinct" restrictively to the values provided.

Let's see it with an example. We are working with this table:

If we create a Selection and do the following filter:

The outcome will be:

As you can see, Id 4 will be excluded, because it's empty.

If we want to keep empty values as well as values distinct to Belgium and France, you will need to add another filter that explicitly includes empty values:

Example: 

In this example, the outcome will be: