It is insecure depending on a few cases*.
Here's a sample of an exploit that a hacker or an above average user can do (with the adb tool and the phone connected to a PC) :-
******May only apply on rooted devices. (not all adb commands require root though) "USB debugging" should also be set prior to using the adb tool*. But as eldarerathis said, some devices can also be rooted through the ADB. In fact, a skilled hacker may also find loopholes for gaining access to the internal data. (perhaps even find a way to utilize the adb without the usb debugging option)
The Android Pattern lock also has many glitches making it relatively unsecure. Here's an example..
While the exploits mentioned above and certain glitches may be fixed by Google in the upcoming versions, a pattern lock remains a pattern lock. ie, It can be considered as a mere solid gateway to using the android's user interface. A pattern lock does not encrypt the contents of a phone. It will not stop a hacker from playing around using the adb shell. He can view the internal memory of the phone etc using the adb tool which is freely provided in the Android SDK
Also, the internal memory is pretty much compromised as there may be other ways to mount it or read it.
All I can say for now is it would be better if you disable "usb debugging" unless you need it, as an added security measure.