Using the experience, refund and run. Here's my thinking.
1. Your concerned about your child and custody. You feel your 100% rights are threatened already for some reason. Legit right there.
2. This man attempted to break into your vehicle with you in it, a woman with legal papers to the land, and a baby. He has no concept of law. Obviously he doesn't care about your views on the matter.
3. You'd be pouring better life money into this pit. Do your payments cover the land taxes? Will it really make your life better now? Or will you be wearing yourself out worried sick? Are you paying good money to be tortured?
4. Putting that money into a small not too old (10 year is the limit with too many places) 'legit' RV and getting some discount camp club memberships would get you well into some years with the kid where CPS wouldn't be so attracted to the child. I'll be cold blooded here. No body wants teens. Babies they'll grab in a heart beat.
5. Get your money back. All of it. Your reason? Your life and that of your precious baby were threatened. You were probably in shock disbelief, now you've parsed the reality and you want your money back.
6. Try city data dot com for a better region to live. It's a wonderful source and forum for all kinds of things from how many doctors of a certain practice (handicap, senior, cancer, thyroid whatever) to what businesses are in the region. Housing and land prices, crime rates, you name it.