Hypercorrection is a common phenomenon. It occurs when someone deliberately tries to avoid making an error in the use of language by overcompensating. As a result, that person is making another error in grammar or style.
The classic example of hypercorrection is the use of “you and I” when “you and me” would actually be correct.
The rule, which is drilled into us from early childhood, is never to use the word “me” in the subject of a sentence. A sentence such as: “You and me are friends” is therefore a no-no. Since this rule was so thoroughly hammered into our heads, many of us still feel uncomfortable about using a construction such as “you and me” anywhere in a sentence.
As a result, a proper sentence such as: “The inheritance will be split between you and me” just does not sound good. When someone mistakenly states: “The inheritance will be split between you and I”, he or she is hypercorrecting.
Another well-worn example of hypercorrection is substituting “whom” for “who” in a sentence like “I need to call my wife, who I know is going to be upset.” Since the rules for using “who” and “whom” are rather tricky and unintuitive, most people will opt for the option that sounds most pretentious.
As a general rule of thumb: it is really OK to start a sentence with “but” or “and”.
Furthermore, please feel free to split infinitives. Your English teacher might be upset, but just keep in mind: language is a tool for communication; and not some lofty scientific goal in itself!