In my UDP server, I'm trying to find out when a client has "disconnected".
Obviously, this makes more sense if I were using a TCP socket but UDP is not "connected".

One way I thought of was having the server constantly send a message to the client,
and if it was not able to contact it, we determine that client to be "disconnected", but this seems
highly inefficient.

Does anyone know the proper way to test if someone is disconnected?

I'm not talking about when the user closes the window, I mean if for any reason the user is actually
NOT in the server (i.e., client crashed, client's internet d/c, etc.)

One way I thought of was having the server constantly send a message to the client,
and if it was not able to contact it, we determine that client to be "disconnected", but this seems
highly inefficient.

This is the only way (it's called "ping-pong" or "heartbeat"), and it's not inefficient, it's only a few bytes every N seconds.

Exactly how often do you think I should be pinging? Or I suppose it wouldn't really matter,
just as long as I'm not using up a ton of bandwidth?

It depends on your needs. It's not really important unless your application needs to do something specific quickly after a disconnection. And don't forget that the total detection time will be greater than the sending period: after sending the packet, you have to wait for the reply, so you need to choose a timeout to decide when the client is considered inactive. And don't forget that UDP is allowed to drop datagrams, so if you want a robust implementation you might need to send multiple ping-pong packets. You can also apply a small optimization: enable the ping-pong messages only when no data is exchanged with the client (if you receive data, you already know that the client is alive).

It's best to not expect to hear anything on a UDP socket.

How many professional games structure their networking is to have two connections to the client; one TCP (for chat and basic, low-cost data transfer) and one for UDP (for everything else that needs to be as fast as possible).

If the TCP side of the connection disconnects, it's safe to assume the UDP connection did too. You could also implement a heartbeat (as Laurent suggested) to make things even safer.