Seems like your OPENSSL_KEYTYPE_RSA would be undefined, get the value
out of somewhere, like a readme or look in the source code. it is
usually an integer. Anyway, the problem might be that your php version
is incompatible with the library etc..
If it also doesnt work with the params, ask the developers who built
the program or whomever it is made/supported by.