Custom Dice Rollers allow you to let the GMTool make rolls with the appropriate dice for your game. For example, the GURPS Dice Roller (the default one) will always roll 3D6 (and add the results toegether).
Installing a custom dice roller works like installing any other plugin: Simply put it in /plugins.