Creating web services with NetBeans

Creating web services with NetBeans

Here I will provide a description of an (finally successful) attempt to create secure web service, using Net Beans and Apache Tomcat. However I will try to describe step by step all the problems that I encountered along the way, and how overcame this obstacles.
So let's begin.

All the pitfalls will be numbered and  marked with brackets [].

First we need to install NetBeans 8 with  Apache Tomcat, and right in the beginning we encounter a curious problem [1] - Tomcat won't start from inside of the Netbeans! Bummer! Not a good start....

[1] To overcome this first obstacle we should set "NO Proxy" in NetBeans Tools->Options->General settings and vuola Tomcat miraculously starts!! (must be  a bug or something...)

OK, so far so good (like in the joke: "I Intend To Live Forever... So Far So good!!")

Next, we add some parameters to tomcat environment, so that we can debug sent - received soap messages (or do you expect everyting to work properly from the 1st attempt?).
So we add at Netbeans ->Servers->  TomCat - > properties -> platform -> VM Options

-Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true
-Dcom.sun.xml.internal.ws.transport.http.HttpAdapter.dump=true

(
  by the way, for a client, there is another variation

 -Dcom.sun.xml.ws.transport.http.client.HttpTransportPipe.dum‌p=true
 -Dcom.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.dump=true

)

If you use standalone Tomcat then for vm-options go to setevn.bat that you create and put besides catalina.bat (tomcat's bin folder)

For example:

rem contents of the setevn.bat 

set JAVA_HOME=C:\Program Files\Java\jdk1.8.0_77

set JAVA_OPTS="-Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true"

"-Dcom.sun.xml.internal.ws.transport.http.HttpAdapter.dump=true"

catalina start -security  (run tomcat with security manager)

To create Tomcat that will run as a service - we have slightly different approach

(set system variable)

set JAVA_HOME=C:\Program Files\Java\jdk1.8.0_77 

Then in dos prompt: 

cd C:\apache-tomcat-8.5.6\bin

service.bat install tomcat8

tomcat8w.exe 

also in java tab / java options set (tight secrity policy): 

-Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true
-Dcom.sun.xml.internal.ws.transport.http.HttpAdapter.dump=true

-Djava.security.manager

-Djava.security.policy=C:\apache-tomcat-8.5.6\conf\catalina.policy

You think we have finished? Not at all!! 

There will be also settings for https, certificates in server.xml , database passwords etc  but all that in due time... for now lets just create an unsecured service.
(by the way I would enable security manager at the very last stage - as there is a load of setting that should be performed in order for the application  to continue to operate with security manager![7])
Also, when deploying the application with security manager, app's context.xml should be copied to tomcat's conf/catalina folder, and renamed to appname.xml - otherwise the application won't deploy [3] (don't ask me why...)

Server 

1. Create new Web Project
2. Import METRO 2.0 library
    (actually you only need webservices-rt.jar to be packaged with final .war file - other libraries are  
    already present in tomcat - and may even cause problems with conflicting loader paths[4])
3. Add xsds that describe your messages
4. Create jax-b xsd binding (right click on packge - new jax-b binding)
5. Java Binding xsd and java code will be generated
6. Make sure in binding xsd's you have (originals may still refer to qualified )
    attributeFormDefault="unqualified"
    elementFormDefault="unqualified" [5]
 
    So that in generated package-info.java there is no reference to  elementFormDefault = javax.xml.bind.annotation.XmlNsForm.QUALIFIED
    Otherwise otherwise extra tag (name space) will be added inside your messages sent by the client
    (ns2:XXXX) and server would not understand the message!!![5]

7. Right click on project-> new -> Web service
8. In design view add operation, as parameters and returning results use object created by the jax-b
    binding procedure

Client

  1. Create new project (java application)
  2. Import METRO 2.0 library  (also may be JAX-WS 2.2.6)
  3. Create a main class
  4. Right click on project-> New -> Web Service Client (give the wsdl url of the web service)
  5. Right click - Insert Code - Web service call (select which operation to call)
  6. Because the generated java code is imperfect a meta tag should be added [6]
      @XmlRootElement(name="XX")
       public static class XX {

       otherwise marshalig /unmarshaling will not work! 

  7.  client input data could be in xml that would be umarshaled and send as parameter of ws call.

  8.  the reply can also be marshaled (if needed)

The unsecured part is ready to be tested! Fingers crossed - hope it works!

Next goal is to secure our communication channel!

To that end, first we should create proper keystores for the client and server, also we need to create truststores for both of them, which will store public keys of entities that are trusted.

Generate server keystore

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -genkey -keyalg RSA -alias mytomcat -dname "CN=Name, OU=Unit, O=Org, L=PlanetEarth, S=MyCity, C=FU" -keystore c:/certs/mytomcatkeystore.jks -validity 36000 -storepass changeit -keypass changeit -ext san=ip:X.X.X.X,ip:127.0.0.1,dns:www.mycompany.com

Export certificate (public key)

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -export -keystore c:/certs/mytomcatkeystore.jks -alias mytomcat -file c:/certs/myserver.cer -storepass changeit

Import into client's trust store 

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -importcert -file e:/certs/myserver.cer -keystore c:/certs/myclinetTrustStore.jks -alias "ClientT" -storepass changeit

Same for the client,

Generate clinent keystore

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -genkey -keyalg RSA -alias ourclient -dname "CN=Name, OU=Unit, O=Org, L=PlanetEarth, S=MyCity, C=FU" -keystore c:/certs/ourclientkeystore.jks -validity 36000 -storepass changeit -keypass changeit

Export certificate (public key)

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -export -keystore c:/certs/ourclientkeystore.jks -alias ourclient  -file c:/certs/ourclinet.cer -storepass changeit

Import into server's trust store 

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -importcert -file e:/certs/ourclinet.cer -keystore c:/certs/mytomcattruststore.jks -alias "TheClient" -storepass changeit

but also

Generate client certificate for web browser

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -importkeystore -srckeystore  c:/certs/myclient.jks -destkeystore c:/certs/myclient1.p12 -srcstoretype JKS -deststoretype PKCS12 - srcstorepass changeit -deststorepass changeit -srcalias MyClient -destalias MyClient1 -srckeypass changeit -destkeypass changeit -noprompt

After creating a key store we should set up tom cat to use it.
So back to tomcat settings.
Go to the tomcat/conf/server.xml

enable 8443 connector:

 <Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
              maxThreads="150" SSLEnabled="true"
 keystoreFile="conf/mytomcatkeystore.jks" clientAuth="true"  scheme="https" secure="true"
 truststoreFile="file:///c:/apache-tomcat-8.5.6/conf/mytomcattruststore.jks" truststorePass="changeit" SSLProtocol="TLSv1.2"
 allowUnsafeLegacyRenegotiation="true"
/>

and also comment out this line [8]

<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />

so that it becomes

<!--<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />-->

also, optionally (if you use database)

edit catalina.properties file, and add to the end of it the following line

mydatabse.password=changeit

so that, in your application's context.xml you could refer to database connection password as

 <Resource name="jdbc/DB" auth="Container"
              type="javax.sql.DataSource" driverClassName="oracle.jdbc.OracleDriver"
              url="jdbc:oracle:thin:@X.X.x.x:1521:xxxx"
              username="myuser" password="${mydatabse.password}" />

And now server is ready. 

On the client side (say you have java client)

the following chunk of code should be ran before ws call

public static void init(){

       System.setProperty("javax.net.ssl.trustStore","c:\\myclinetTrustStore.jks");

       System.setProperty("javax.net.ssl.trustStorePassword","changeit"); 

       System.setProperty("https.protocols", "TLSv1.2");

       System.setProperty("javax.net.ssl.keyStore","c:\\myclient.jks"); 

       System.setProperty("javax.net.ssl.keyStorePassword","changeit"); 

       System.setProperty("javax.net.debug","ssl,record,keygen,handshake"); 

    }

Code generation with python

Here I will preset a small code generator I am using to generate java, html, jsp, xsd and other codes.

I am using Python27 and Jinja2-2.7.2 

My Files:  

InputData.txt

:This is field list
field1:field1 description
field2:field2 description
field3:field3 description
field4:field4 description
field5:field5 description
field6:field6 description
field7:field7 description

CodeGenerator.py

import jinja2
import codecs
templateLoader = jinja2.FileSystemLoader( searchpath="." )
templateEnv = jinja2.Environment( loader=templateLoader )

TEMPLATE_FILE = "CodeGenerator.jinja"
template = templateEnv.get_template( TEMPLATE_FILE )


COLUMNS       = [tuple(line.split(':')) for line in codecs.open( "InputData.txt", "r", "utf-8" )]

COLUMNS       = map(lambda s: (s[0],(s[0].strip().title(),s[1].strip())), COLUMNS)
#title() copy of the string in which first characters of all the words are capitalized.
#strip()  copy of the string, all chars have been stripped from the beginning and the end
#lambda s --> (field1,(Field1,field1 description))

#ignore the first line
COLUMNS.pop(0)

#add variables to work with
templateVars = { "table" : "MyTableName",
                 "description" : "A simple code generator",
                 "columns" : COLUMNS
               }

outputText = template.render( templateVars )
f = open('Generated.txt', 'w')
outputText = outputText.encode('utf-8')
f.write(outputText)
f.close()
print outputText

CodeGenerator.jinja

//------------------model------------------------

//Generated code for model 

import java.io.Serializable;

public class {{table|capitalize}}_Model implements Serializable{

{%for columnName,columnTitle in columns%}

private String {{columnName|lower}};{%endfor%}

{% for columnName,columnTitle in columns %}

public void set{{columnName|capitalize}}(String {{columnName|lower}}){ 

  this.{{columnName|lower}}={{columnName|lower}};

}

public String get{{columnName|capitalize}}(){ 

  return this.{{columnName|lower}};

}

{% endfor %}

public {{table|capitalize}}_Model({% for columnName,columnTitle in columns %}String {{columnName|lower}}{% if not loop.last %},{% endif %}{% endfor %}){

        {% for columnName,columnTitle in columns %}

this.{{columnName|lower}}={{columnName|lower}}; {% endfor %}

    }

public String toString(){

return {% for columnName,columnTitle in columns %}"{{columnName|lower}}:" + this.{{columnName|lower}}{% if not loop.last %}+{% endif %}{% endfor %};

}

}

//------------------model------------------------

//

//------------------jsp example-----------------

{% for columnName,columnTitle in columns %}

<s:label  cssStyle ="margin-top:5px;color:blue;" value="{{columnTitle[1]}} :" id="{{columnName}}_label"/>

<s:textfield label="{{columnTitle[1]}}" name="myBean.{{columnName}}" />

{% endfor %}

//------------------jsp example-----------------