Problem one is that you have a manual.... they are dead off the line, drive an auto version and you'll think it was a different engine. They are also apparently detuned to "spare" the gearbox and fragile clutch.
In your current situation gearing is your friend, early diesels came with 4.1's so your not doing something bizarre. However depending on your driving style the change could hurt your economy...... its not a perfect world
it will help fill the off idle hole a tiny bit but not by much.
Of course ratio changes are expensive, so think about future tire upgrades now.. if your looking at bigger rubber later on then 4.1 might not be enough..... and you'll end up doing it all again.
A shift to a higher numerical ratio will also help off road if that's important to you.