What does it mean for a star to go nova or supernova? Can I safely observe these?
It's a matter of size and stellar evolution. There are many, many types of stellar explosions. The University of Arizona has one page that describes these types. generally, a Novs is not what we think of (i.e. a star exploding). That's actually a Type II Supernova.
According to that site:
Novae are frequently (perhaps always) members of binary systems where the more massive star evolves more quickly and becomes a white dwarf. The less massive star loses some of its hydrogen-rich outer atmosphere to the white dwarf where it collects. The high gravity and intense heat of the white dwarf make conditions suitable for hydrogen fusion into helium. When enough hydrogen collects, it explodes in a fusion reaction, causing a nova. The gas ejected in the explosion moves outwards at 1000-2000 km/sec.
The Supernova has two distinct types:
The "Type I" supernovae arise when the white dwarf star member of a binary system accretes so much matter from its companion that it is tipped over the 1.4M Chandrasekar limit. The white dwarf collapses very rapidly (just a few seconds) until the infalling layers hit the very hard surface of the neutron star. The resulting shock goes back through the collapsing layers, and they explode by hydrogen fusion.
And the one we are more familiar with from popular media:
The "Type II" supernovae are the result of a massive star consuming all of its nuclear fuel and then exploding. The black line traces the evolution of a massive star schematically, until it reaches the supernova stage.
Type I: supernovae WITHOUT hydrogen absorption lines in their spectrum
Type II: supernovae WITH hydrogen absorption lines in their spectrum.
The type II supernovae are massive stars whose iron cores collapse and then rebound, shock heating the outer layers of the star, which then explode outward.
The type I supernovae are subdivided into three subclasses, called (boringly enough), Type Ia, Type Ib, and Type Ic.
Type Ia: no hydrogen lines, no helium lines, strong silicon lines
Type Ib: no hydrogen lines, strong helium lines
Type Ic: no hydrogen lines, no helium lines, no silicon lines
Type Ib and type Ic supernovae are massive stars which lost their outer layers in a stellar wind before core collapse.
Type Ib supernovae lost their hydrogen-rich outer layer, revealing the helium-rich layer immediately below.
Type Ic supernovae suffered more mass loss as supergiants, losing both the hydrogen-rich layer and the helium-rich layer (revealing the carbon-rich layer below).
Type Ib and type Ic supernovae are essentially the same as type II supernovae.
As for observing them, since none of them are expected to be close to us (i.e. no stars near us able to go supernova), it should be safe. The problem is predicting when one will happen. For instance, Betelgeuse is expected to go supernova "soon", but in stellar terms, that could be well over 100,000 years from now. The list of candidates in the Milky Way doesn't bode well for one in your lifetime. Although astronomers are often seeing them in other galaxies, and some are observable by amateur astronomers. In fact, just this past August (2013) an amateur discovered a nova.