Wednesday, December 13, 2017

Showing build time, version etc in web apps and desktop applications


It is quite convenient to be able to show Application Version and build time in any given java program. Below is my way of doing this. 

Ant script for War Files 
<target name="-post-dist">      
        <!-- Empty placeholder for easier customization. -->
        <!--<property file="build.properties"/>-->
        <buildnumber file="build.version"/>
        <tstamp>
            <format property="timestamp" pattern="yyyy-MM-dd HH:mm:ss"/>
        </tstamp>
            <jar destfile="${dist.war}" update="true">
     
        <manifest>
            <attribute name="Manifest-Version" value="1.0" />        
            <attribute name="Created-By" value="Me" />        
            <attribute name="Bundle" value="${build.number}"/>
            <attribute name="Bundle-Date" value="${timestamp}" />
        </manifest>
        </jar>
        <!-- You can override this target in the ../build.xml file. -->
 </target>
Java Code for Web Application 

 public static String getBuildNumberAndDateFromWar(WebServiceContext context){
        String buildNumber="";
        try {
            javax.xml.ws.handler.MessageContext mc = context.getMessageContext();
            HttpServletRequest req = (HttpServletRequest) mc.get(javax.xml.ws.handler.MessageContext.SERVLET_REQUEST);
            InputStream stream = req.getServletContext().getResourceAsStream("/META-INF/MANIFEST.MF");
            Manifest manifest = new Manifest(stream);
            Attributes attributes = manifest.getMainAttributes();
            String bundle = attributes.getValue("Bundle");
            String bundle_date = attributes.getValue("Bundle-Date");
            stream.close();
            if (bundle!=null && !"null".equals(bundle))
                buildNumber = " [bundle:"+ bundle + ", date:" + bundle_date+"] ";

        } catch (Exception e) {
            e.printStackTrace();
        }
        return buildNumber;
    }

Ant Script for Jar file

 <target name="-post-jar">
        <!-- Empty placeholder for easier customization. -->
        <!-- You can override this target in the ../build.xml file. -->
        <buildnumber file="build.version"/>
        <tstamp>
            <format property="timestamp" pattern="yyyy-MM-dd HH:mm:ss"/>
        </tstamp>
            <jar destfile="${dist.jar}" update="true">
     
        <manifest>
            <attribute name="Manifest-Version" value="1.0" />        
            <attribute name="Version" value="1.0.1" />
            <attribute name="Created-By" value="Me" />        
            <attribute name="Bundle" value="${build.number}"/>
            <attribute name="Bundle-Date" value="${timestamp}" />
        </manifest>
        </jar>
 </target>

Java code for Java Application 

  public static  String getVersionBuildNumberAndDateFromJar(){
        String buildNumber="";
        try {
            InputStream stream = (new EscrowPaymentsAppClient()).getClass().getResourceAsStream("/META-INF/MANIFEST.MF");
            Manifest manifest = new Manifest(stream);
            Attributes attributes = manifest.getMainAttributes();
            String version = attributes.getValue("Version");
            String bundle = attributes.getValue("Bundle");
            String bundle_date = attributes.getValue("Bundle-Date");
            stream.close();
            if (bundle!=null && !"null".equals(bundle))
                buildNumber = " [version:"+ version + ", bundle:"+ bundle + ", date:" + bundle_date+"] ";

        } catch (Exception e) {
            e.printStackTrace();
        }
        return buildNumber;
    }

Tuesday, December 12, 2017

Using Python's pip behind Firewall (on Windows)

To use python behind a firewall that uses ntlm authentication we will use ntlm maps local proxy.

So here are the steps:

1. Download and install ntlm maps 
   (local proxy that will handle the authentication with our corporate proxy)
    http://ntlmaps.sourceforge.net/

2. Configure server.cfg of the ntlm maps,  change the following
      LISTEN_PORT:5865
      PARENT_PROXY:XXXX.XXXX.XXXX.XXXX
      PARENT_PROXY_PORT:XXXX
      NT_DOMAIN:xxxxx.xxx
      USER:xxxxxxxxxxx
      PASSWORD: 

   If we leave the password blank it will ask us for it at run-time.

3. if ntlm maps does not run - then some dlls are missing those are
    -MSVCP71.DLL
    -MSVCR71.DLL
    Copy then into SysWOW64 if needed (on 64 bit system) or into (System32 on 32 bit system)

4. run the ntlm maps

5.  download and install python
       https://www.python.org/downloads/
       https://www.python.org/ftp/python/2.7.14/python-2.7.14.msi

6. open command prompt (cmd) and enter:
      set http_proxy=localhost:5865

7. use pip to download and install plugins
    -cx_Oracle-doc
    pip --proxy http://127.0.0.1:5865 install cx_Oracle --upgrade
    -Jinja2-2.7.2
    pip --proxy http://127.0.0.1:5865 install Jinja2 --upgrade
    -mechanize-0.2.5
    pip --proxy http://127.0.0.1:5865 install mechanize --upgrade
    -beautifulsoup4
    pip --proxy http://127.0.0.1:5865 install beautifulsoup4 --upgrade

Windows 10 God Mode Folder on Desktop

If you have created a "God Mode" folder on your desktop - think twice before deciding to keep it there.
It turns out that some applications will refuse to perform their basic functionalities (like: open file, browse etc) and will crash - only because the folder sits on your desktop.


As always, had to learn that the hard way....

*If you don't know what the "God Mode" folder is - then probably you don't have one . This folder  usually is created explicitly by the user using some undocumented features of the Windows OS.




Windows 10 missing sticky notes and microsoft store problem

After installing Windows 10 I realized that the Microsoft Store was missing also the sticky note app was not there either...

The solution came from power-shell.
I guess Microsoft has adopted linux-like approach in getting updates and missing packages.

So here what I did:

 1. Enter Windows Power-Shell (as Admin)
 2. Get-Appxpackage -Allusers > res.txt
 3. Add-AppxPackage -register "C:\Program Files\WindowsApps\[packageFullName]\AppxManifest.xml" -DisableDevelopmentMode

Here the res.txt file is needed to find missing package's full name.

[Microsoft.MicrosoftStickyNotes]
[Microsoft.WindowsStore]

We use that name to download and install the package (step 3).

Tuesday, April 4, 2017

Creating web services with NetBeans

Creating web services with NetBeans

Here I will provide a description of an (finally successful) attempt to create secure web service, using Net Beans and Apache Tomcat. However I will try to describe step by step all the problems that I encountered along the way, and how overcame this obstacles.
So let's begin.

All the pitfalls will be numbered and  marked with brackets [].

First we need to install NetBeans 8 with  Apache Tomcat, and right in the beginning we encounter a curious problem [1] - Tomcat won't start from inside of the Netbeans! Bummer! Not a good start....

[1] To overcome this first obstacle we should set "NO Proxy" in NetBeans Tools->Options->General settings and vuola Tomcat miraculously starts!! (must be  a bug or something...)

OK, so far so good (like in the joke: "I Intend To Live Forever... So Far So good!!")

Next, we add some parameters to tomcat environment, so that we can debug sent - received soap messages (or do you expect everyting to work properly from the 1st attempt?).
So we add at Netbeans ->Servers->  TomCat - > properties -> platform -> VM Options

-Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true
-Dcom.sun.xml.internal.ws.transport.http.HttpAdapter.dump=true

(
  by the way, for a client, there is another variation

 -Dcom.sun.xml.ws.transport.http.client.HttpTransportPipe.dum‌p=true
 -Dcom.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.dump=true

)

If you use standalone Tomcat then for vm-options go to setevn.bat that you create and put besides catalina.bat (tomcat's bin folder)

For example:

rem contents of the setevn.bat 
set JAVA_HOME=C:\Program Files\Java\jdk1.8.0_77
set JAVA_OPTS="-Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true"
"-Dcom.sun.xml.internal.ws.transport.http.HttpAdapter.dump=true"



catalina start -security  (run tomcat with security manager)

To create Tomcat that will run as a service - we have slightly different approach

(set system variable)
set JAVA_HOME=C:\Program Files\Java\jdk1.8.0_77 

Then in dos prompt: 

cd C:\apache-tomcat-8.5.6\bin
service.bat install tomcat8

tomcat8w.exe 

also in java tab / java options set (tight secrity policy): 
-Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true
-Dcom.sun.xml.internal.ws.transport.http.HttpAdapter.dump=true
-Djava.security.manager
-Djava.security.policy=C:\apache-tomcat-8.5.6\conf\catalina.policy

You think we have finished? Not at all!! 

There will be also settings for https, certificates in server.xml , database passwords etc  but all that in due time... for now lets just create an unsecured service.
(by the way I would enable security manager at the very last stage - as there is a load of setting that should be performed in order for the application  to continue to operate with security manager![7])
Also, when deploying the application with security manager, app's context.xml should be copied to tomcat's conf/catalina folder, and renamed to appname.xml - otherwise the application won't deploy [3] (don't ask me why...)

Server 

1. Create new Web Project
2. Import METRO 2.0 library
    (actually you only need webservices-rt.jar to be packaged with final .war file - other libraries are  
    already present in tomcat - and may even cause problems with conflicting loader paths[4])
3. Add xsds that describe your messages
4. Create jax-b xsd binding (right click on packge - new jax-b binding)
5. Java Binding xsd and java code will be generated
6. Make sure in binding xsd's you have (originals may still refer to qualified )
    attributeFormDefault="unqualified"
    elementFormDefault="unqualified" [5]
 
    So that in generated package-info.java there is no reference to  elementFormDefault = javax.xml.bind.annotation.XmlNsForm.QUALIFIED
    Otherwise otherwise extra tag (name space) will be added inside your messages sent by the client
    (ns2:XXXX) and server would not understand the message!!![5]

7. Right click on project-> new -> Web service
8. In design view add operation, as parameters and returning results use object created by the jax-b
    binding procedure

Client

  1. Create new project (java application)
  2. Import METRO 2.0 library  (also may be JAX-WS 2.2.6)
  3. Create a main class
  4. Right click on project-> New -> Web Service Client (give the wsdl url of the web service)
  5. Right click - Insert Code - Web service call (select which operation to call)
  6. Because the generated java code is imperfect a meta tag should be added [6]
      @XmlRootElement(name="XX")
       public static class XX {
       otherwise marshalig /unmarshaling will not work! 
  7.  client input data could be in xml that would be umarshaled and send as parameter of ws call.
  8.  the reply can also be marshaled (if needed)


The unsecured part is ready to be tested! Fingers crossed - hope it works!

Next goal is to secure our communication channel!

To that end, first we should create proper keystores for the client and server, also we need to create truststores for both of them, which will store public keys of entities that are trusted.

Generate server keystore

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -genkey -keyalg RSA -alias mytomcat -dname "CN=Name, OU=Unit, O=Org, L=PlanetEarth, S=MyCity, C=FU" -keystore c:/certs/mytomcatkeystore.jks -validity 36000 -storepass changeit -keypass changeit -ext san=ip:X.X.X.X,ip:127.0.0.1,dns:www.mycompany.com

Export certificate (public key)

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -export -keystore c:/certs/mytomcatkeystore.jks -alias mytomcat -file c:/certs/myserver.cer -storepass changeit

Import into client's trust store 

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -importcert -file e:/certs/myserver.cer -keystore c:/certs/myclinetTrustStore.jks -alias "ClientT" -storepass changeit

Same for the client,

Generate clinent keystore

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -genkey -keyalg RSA -alias ourclient -dname "CN=Name, OU=Unit, O=Org, L=PlanetEarth, S=MyCity, C=FU" -keystore c:/certs/ourclientkeystore.jks -validity 36000 -storepass changeit -keypass changeit

Export certificate (public key)

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -export -keystore c:/certs/ourclientkeystore.jks -alias ourclient  -file c:/certs/ourclinet.cer -storepass changeit

Import into server's trust store 

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -importcert -file e:/certs/ourclinet.cer -keystore c:/certs/mytomcattruststore.jks -alias "TheClient" -storepass changeit

but also

Generate client certificate for web browser

"c:/Program Files/Java/jdk1.8.0_77/bin/keytool.exe" -importkeystore -srckeystore  c:/certs/myclient.jks -destkeystore c:/certs/myclient1.p12 -srcstoretype JKS -deststoretype PKCS12 - srcstorepass changeit -deststorepass changeit -srcalias MyClient -destalias MyClient1 -srckeypass changeit -destkeypass changeit -noprompt

After creating a key store we should set up tom cat to use it.
So back to tomcat settings.
Go to the tomcat/conf/server.xml

enable 8443 connector:

 <Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
              maxThreads="150" SSLEnabled="true"
 keystoreFile="conf/mytomcatkeystore.jks" clientAuth="true"  scheme="https" secure="true"
 truststoreFile="file:///c:/apache-tomcat-8.5.6/conf/mytomcattruststore.jks" truststorePass="changeit" SSLProtocol="TLSv1.2"
 allowUnsafeLegacyRenegotiation="true"
/>

and also comment out this line [8]

<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />

so that it becomes

<!--<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />-->

also, optionally (if you use database)

edit catalina.properties file, and add to the end of it the following line

mydatabse.password=changeit

so that, in your application's context.xml you could refer to database connection password as

 <Resource name="jdbc/DB" auth="Container"
              type="javax.sql.DataSource" driverClassName="oracle.jdbc.OracleDriver"
              url="jdbc:oracle:thin:@X.X.x.x:1521:xxxx"
              username="myuser" password="${mydatabse.password}" />

And now server is ready. 

On the client side (say you have java client)

the following chunk of code should be ran before ws call

public static void init(){
       System.setProperty("javax.net.ssl.trustStore","c:\\myclinetTrustStore.jks");
       System.setProperty("javax.net.ssl.trustStorePassword","changeit"); 
       System.setProperty("https.protocols", "TLSv1.2");
       System.setProperty("javax.net.ssl.keyStore","c:\\myclient.jks"); 
       System.setProperty("javax.net.ssl.keyStorePassword","changeit"); 
       System.setProperty("javax.net.debug","ssl,record,keygen,handshake"); 
    }

Code generation with python

Here I will preset a small code generator I am using to generate java, html, jsp, xsd and other codes.

I am using Python27 and Jinja2-2.7.2 

My Files:  

InputData.txt

:This is field list
field1:field1 description
field2:field2 description
field3:field3 description
field4:field4 description
field5:field5 description
field6:field6 description
field7:field7 description

CodeGenerator.py

import jinja2
import codecs
templateLoader = jinja2.FileSystemLoader( searchpath="." )
templateEnv = jinja2.Environment( loader=templateLoader )

TEMPLATE_FILE = "CodeGenerator.jinja"
template = templateEnv.get_template( TEMPLATE_FILE )


COLUMNS       = [tuple(line.split(':')) for line in codecs.open( "InputData.txt", "r", "utf-8" )]

COLUMNS       = map(lambda s: (s[0],(s[0].strip().title(),s[1].strip())), COLUMNS)
#title() copy of the string in which first characters of all the words are capitalized.
#strip()  copy of the string, all chars have been stripped from the beginning and the end
#lambda s --> (field1,(Field1,field1 description))

#ignore the first line
COLUMNS.pop(0)

#add variables to work with
templateVars = { "table" : "MyTableName",
                 "description" : "A simple code generator",
                 "columns" : COLUMNS
               }

outputText = template.render( templateVars )
f = open('Generated.txt', 'w')
outputText = outputText.encode('utf-8')
f.write(outputText)
f.close()
print outputText

CodeGenerator.jinja

//------------------model------------------------
//Generated code for model 
import java.io.Serializable;
public class {{table|capitalize}}_Model implements Serializable{
{%for columnName,columnTitle in columns%}
private String {{columnName|lower}};{%endfor%}

{% for columnName,columnTitle in columns %}
public void set{{columnName|capitalize}}(String {{columnName|lower}}){ 
  this.{{columnName|lower}}={{columnName|lower}};
}
public String get{{columnName|capitalize}}(){ 
  return this.{{columnName|lower}};
}
{% endfor %}
public {{table|capitalize}}_Model({% for columnName,columnTitle in columns %}String {{columnName|lower}}{% if not loop.last %},{% endif %}{% endfor %}){
        {% for columnName,columnTitle in columns %}
this.{{columnName|lower}}={{columnName|lower}}; {% endfor %}
    }
public String toString(){

return {% for columnName,columnTitle in columns %}"{{columnName|lower}}:" + this.{{columnName|lower}}{% if not loop.last %}+{% endif %}{% endfor %};

}

}
//------------------model------------------------
//
//------------------jsp example-----------------
{% for columnName,columnTitle in columns %}
<s:label  cssStyle ="margin-top:5px;color:blue;" value="{{columnTitle[1]}} :" id="{{columnName}}_label"/>
<s:textfield label="{{columnTitle[1]}}" name="myBean.{{columnName}}" />
{% endfor %}
//------------------jsp example-----------------