What's the difference between a Group and a Tag?
We see Tags as a way of describing / naming a thing and Groups as a way of creating a set of things (which may or may not have things in common).
Very concretely, in the kitchen you might have a thing we would Tag as a 'Refrigerator', and a 'Stove', and a 'Sink'. You can you combine these objects into a set using the Group concept (they are members of 'Kitchen Group').
On another level, Groups have 'members' and their membership status history is generally tracked by the system (added, pending confirmation, removed/opted-out).
Groups automatically become an access control (permissionable) object. You can also do this with Tags by creating a smart group which searches for contacts assigned that tag but this is less efficient.
You can selectively expose Groups via Profile forms (sign-up/user registration/my account) by including Groups in the Profile and setting 'visibility' at the Group level. So you can have some Groups that 'the public' can see and join, and others that are hidden.
On implementation level:
You might also want to check out this faq entry Groups, Tags, Custom Fields, Relationships: What do I use for what?