The short answer:
I would simply reduce the power limit in MSI afterburner until the temperature / noise reach the desired limit (you could use some similar GPU tweaking tool - however this one works fine with other brands also).
The TLDR part:
Overclocking and underclocking is the same thing with similar iteration of the ideal frequency / voltage combination, only their purpose is different (maximum performance vs maximum efficiency). So if you want to refine your underclocking further, then you can reduce manually the voltage and/or the GPU frequency with this tool. However I think that the GPUs have their own frequency / voltage and temperature / power consumption tables in their BIOS, so decreasing voltage will "automatically" decrease frequency to maintain the stability of the chip.
Regarding testing: You can pre-evaluate your settings by furmark or similar tools, but the only sure thing is the GPUGrid application itself. You should monitor the progress of your workunits closely, changing only one GPU parameter at a time, taking notes every time the time and date and of what you've changed and what else changed "automatically". As different workunit batches stress the GPU to different extent, one setting may not work for another batch, so if a new batch is failing, you should probably increase voltage, or more likely decrease frequency (just as when overclocking).
Note that power consumption is in square ratio with voltage, and in direct ratio with frequency. E.g. decreasing voltage by 10% without reducing the frequency saves more energy (21%) than decreasing frequency by 10% without reducing the voltage. As far as I know this two parameters are linked for stability reasons, therefore in practice you can't reduce one of them without the other being reduced automatically - the another consequence of this behavior is that you don't have to figure out that voltage vs stability curve as it is programmed into the BIOS.