I figured out why people who don't know you are harsher.

It's because they don't know you well and are bound to seem more harsh.