I realize many verbs add the "be" to the beginning, which fundamentally changes the meaning, e.g., kommen and bekommen, or stehen and bestehen. However, it seems as though the "be" in belehren doesn't change the meaning much, if at all. So when should I use it?
I disagree with you that it doesn't change the meaning that much...
jemanden etwas lehren
means to teach something to somebody.
So you'd be building up new knowledge by teaching them.
in most cases means that you're correcting somebody who thinks he knows, but is not really correct.
So you'd be straightening his facts.
One other use of belehren would be that you inform somebody of something.
An example would be if you tell them about their rights ("You have the right to remain silent...")
In that case it would be
jemanden über seine Rechte belehren
So to sum up, with lehren you're kind of teaching or providing knowledge to somebody.
With belehren you're rather telling somebody something than you're teaching.
So you're doing that probably with more resolution and authority.
There is a grammatical difference and a difference in meaning...
Belehren takes a person as a direct object in accusative case. If you want to include whatever is being taught that is done using the preposition über or hinsichtlich
Ich belehre den Mann über die Sache.
Belehren has a notion of completion... at least more than lehren has. After having been belehrt, I expect the person to know everything about the matter. Also, belehren is done with rather limited fields of knowledge. It is one specific thing that is being taught... like a code of conduct or the late fees of the library. So you can't really belehren someone about biology... I wouldn't know how to understand the following sentence:
Ich belehre dich über Biologie.
This is so vague that it could be anything... but it certainly is not the biological knowledge itself. Belehren also has a negative touch to it... I wouldn't appreciate being belehrt. I'd prefer to be informiert.
Lehren can take 2 direct accusative objects... the thing being taught and the person being taught.
Ich lehre Klavier.
Der Professor hat mich die Liebe gelehrt.
Lehren doesn't imply completion. It can but it doesn't have to. It can be a life long process. It is also the word for the profession... to teach. Also lehren is not negative sounding. It is pretty neutral. In daily life, I think lehren as a verb is on the decline. I find it kind of weird in a sentence and I would opt for unterrichten. However, the nouns derived from lehren are totally part of everyday talk. Lehrer, Lehrling, Lehrplan, Lehrbuch, Lehrfilm, Lehrvideo, Lehrgang...
Could it be that "belehren" is "(trying) to teach without being asked to"?
"Belehren" can easily be really unwanted and make the person exposed to it feel annoyed, as there is no desire to receive that kind of 'teaching'. It's actually the definition of "spam"...
However, like in a case of "Rechtsbelehrung" for example, it might be very useful to the one that gets "belehrt", even though they may not like it very much in the moment.