I have downloaded the jsp4 package from layer1.cpanel.net, and run the install script. It took a few minutes, and then I added the requested lines into the config file. I also have suexec enabled as I've seen posted somewhere, although I'm not sure it's related.
Now with those entries the web server starts just fine, but I wrote a quick test jsp file, but it does not get processed as a jsp file, but just spit out as html.
Are there any further steps to allow .jsp files to be properly handled?
For those wondering how to make it work, after installing the above setup, what noone mentions is that you must enable servlet/jsp support on a per-host instance using WHM. It's under the Plugins section on the left hand side.