I usually just use normal bandits and uncheck their "attack players". They'll still fight against aggro mobs, and you can use sequences to make them act somewhat like a 'bandit camp', though not quite as pretty.
There's no way to make them attack some NPCs but not others though. The AI is set in stone in that regard. Bandits will always still fight civilians, guards, kami and karavan. Unless you set them to not fight any mobs at all.
Setting aggro range to 0 will make them not attack anything, but still fight back against anything that attacks them. They'll not help a player that storms into their village with a kincher in tow, though, until the kincher is done with the player and turns on them.
