How do I restrict that the switch only can be accessed by specific IP address and protocol?

You can try to configure the “Access Management Configuration” to restrict IP address range and protocol.

Web UI path:

Security —> Access Management —> Configuration

CLI command: (Suppose only allow IP address 192.168.3.30~192.168.3.50 can access the switch by HTTP/HTTPS protocol .)

Switch# access

Switch(access)# add 1 ipv4 192.168.3.30 192.168.3.50 web