Changing the context name in Tomcat

Sometimes, it may happen, you want to provide the application context name  different from the WAR file name while deploying in Tomcat.

Scenario:

WAR Name: ABC.war
The default url becomes: http://localhost:8080/ABC

Want to access the url as: http://localhost:8080/XYZ

There are 2 options available to do this change

Option #1:
Here is the steps that can be followed:

1. Navigate to <TOMCAT_HOME>/conf/server.xml
2. Goto the Host section & do the below changes
3. Change it to below:

 <Host name="localhost"  appBase="webapps"
      unpackWARs="false" autoDeploy="false" deployOnStartup="false">    

Please note, unpackWARs , deployOnStartup, autoDeploy all three should be marked as false, else 2 folders will be generated one with name ABC & another XYZ.

4. Add the context changes under Host

 <Host name="localhost"  appBase="webapps"
      unpackWARs="false" autoDeploy="false" deployOnStartup="false">  
<Context path="/XYZ" docBase="ABC.war"/>
 <!-- other preexisting configuration-->
<Host>
5. Start Tomcat going to <TOMCAT_HOME>/bin/startup.bat (windows) or <TOMCAT_HOME>/bin/startup.sh in Linux environment.

Option#2:

1. Explode the ABC.war (unzip the WAR file)
2. Place the exploded WAR in a folder outside Tomcat Directory (e.g. D:\mywebapps)
3. So, now the exploded WAR path will be D:\mywebapps\ABC
4. Create an xml file in <TOMCAT_HOME>/conf/Catalina/localhost named XYZ.xml (the name of expected context)
5. Now add the below line in XYZ.xml
6. <Context path="/XYZ" docBase="D:/mywebapps/ABC"/>
7. docBase refer to the path where exploded WAR is placed.
8. Start Tomcat going to <TOMCAT_HOME>/bin/startup.bat (windows) or <TOMCAT_HOME>/bin/startup.sh in Linux environment.
   
   

Thread Dump in Java

Steps to take thread dump in Java in Windows

1. Download PSTools from below link
https://docs.microsoft.com/en-us/sysinternals/downloads/pstools
2. Use below command to take thread dump:

psexec -s <Path_to_JDK_bin_folder>\jstack.exe -l <process_id> ><PATH_WITH_FILE_NAME_FOR_DUMP>

e.g.

psexec -s D:\jdk1.8.0_171\bin\jstack.exe -l 319732 >D:\dump.txt

3. Download IBM Thread Dump Analyzer(TDA) from below link

https://public.dhe.ibm.com/software/websphere/appserv/support/tools/jca/jca465.jar

4. Double click on the jar to open

5. Click on File-->Open Thread Dumps.

6. Choose the thread dump file

7. Click on Analysis-->Thread Status Analysis

 

Details on TDA can be found here 

https://www.ibm.com/support/pages/ibm-thread-and-monitor-dump-analyzer-java-tmda

https://public.dhe.ibm.com/software/websphere/appserv/support/tools/jca/

Useful Production Profiling links

Few days ago I was looking for a Profiling to be done on Production environment. Earlier, I used to use JProfiler & JavaMelody for performance / issue debugging in production environment.

But , of late I found Alibaba has created a new profiling tool for production usage. The below link contains the details. Please check out. Seems interesting

https://medium.com/@Alibaba_Cloud/troubleshooting-production-issues-with-alibabas-arthas-68d8ec2824d7

Useful Information on Application Security

1. Sql Injection Checking Library:
https://github.com/rkpunjal/sql-injection-safe

2. Checking Vulnerable libraries in application:

Add below plugin in pom.xml of you application. This will provide the list of libraries which are vulnerable. A file named dependency-check-report.html will be generated in target folder of you maven based app
 <plugin>
              <groupId>org.owasp</groupId>
              <artifactId>dependency-check-maven</artifactId>
              <version>5.2.4</version>
              <executions>
                  <execution>
                      <goals>
                          <goal>check</goal>
                      </goals>
                  </execution>
              </executions>
            </plugin>
More details can be found in below link:
https://jeremylong.github.io/DependencyCheck/dependency-check-maven/

3. Security Guidelines Tutorial:

https://code.likeagirl.io/pushing-left-like-a-boss-part-1-80f1f007da95

4. Code Review Checklist
https://github.com/softwaresecured/secure-code-review-checklist

5. Burp Extension:

https://github.com/snoopysecurity/awesome-burp-extensions

Http Client Code Auto Generation

Many a times we write http client code in various programming language by our own. Postman (a Chrome Browser extension), provides an way to auto-generate the HTTP client code. Below are the steps to proceed:

1. Open the Postman extension from Chrome
2. Hit the url you wnat to develop the client code
3. Provide other details in Authorization/Header tabs
4. Provide the Content in Body tab
5. Click on the Code link in Right Side.
6. You will be provided with list of options with Programming language like Java/Python etc.
7. Choose the option & your code is there.
8. You can now add the code in your application with the library used.

Happy Coding !
  

SQL Injection testing using SqlMap & Postman

SqlMap is very powerful tool for Automated Sql Injection testing for Web Application/API (SOAP/REST). This blog describes the procedure to get started with testing



SqlMap & Python Installation:

1.Download Python 2.7.16.
2.Goto https://www.python.org/downloads/release/python-2716/
3.Choose Windows x86-64 MSI installer option for Windows Installation
4.Add the folder where Python is installed in Path (Environment variable). e.g. If Python is installed in C:\Python27 then add this path in Path Variable in Windows
5.Download the .Zip version of SQLMap from http://sqlmap.org/
6.Extract in any folder in any Drive (e.g. D:\sqlmapproject)

Preparation of Test Data:
Here we are going to test Sql Injection in url http://testphp.vulnweb.com/listproducts.php?cat=1
1.Open Chrome Browser
2.Open Postman extension in Chrome. Install from Chrome Web Store if Postman is not installed
3.Hit the url mentioned above using GET request
4.Click on the Right Side of Postman in Link "Code"
5,Choose Http Option.
6.Copy the content & paste in a text file (e.g. attack.txt)

Sql Injection Testing:


1.Open Windows Command prompt
2.Navigate to the folder where SqlMap is extracted (D:\sqlmapproject)
3.Copy the attack.txt in D:\sqlmapproject
4.Run below command. adding --flush-session --fresh-queries will enable to execute the test cases freshly; else the old cached data will be shown in command prompt.
python sqlmap.py -r attack.txt --dbs --flush-session --fresh-queries
This will run all the sql injection test cases automatically & provide the output




Using the same way REST/SOAP API can be tested

Notes: In Windows 10, you might get an error Python not installed & need to install from Microsoft Store. In that case, declare the Python installation path at the top as below:

JavaMelody Report generation issue

Recently I have upgraded some of my application in Tomcat 9 from Tomcat 6. After migration, I found the Java Melody report is not generating properly. The sql statistics were not coming.


Upon investigating, I found the JavaMelody Listener should be ordered first among other listeners. As I was using Spring, hence Spring context Listener has to come in 2nd place & Java Melody Session Listener should come first in order.


Below is the snapshot of web.xml with ordering configuration:


Required JARs:
itext-2.1.7.jar

javamelody-core-1.42.0.jar

jrobin-1.5.9.jar



Url format to access Java Melody Report:


http://<Host>:<port>/<ContextRoot>/monitoring

Web.xml structure with ordering configuration:

<?xml version="1.0" encoding="ISO-8859-1"?>

<web-app version="3.0" xmlns="http://java.sun.com/xml/ns/javaee"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://java.sun.com/xml/ns/javaee

http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"

metadata-complete="true">

<context-param>
<param-name>contextConfigLocation</param-name>
<param-value>/WEB-INF/applicationContext.xml</param-value>
</context-param>

<filter>
<filter-name>monitoring</filter-name>
<filter-class>net.bull.javamelody.MonitoringFilter</filter-class>
</filter>

<filter-mapping>
<filter-name>monitoring</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

<listener>
<listener-class>net.bull.javamelody.SessionListener</listener-class>
</listener>
<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>


<!-- Other servlet details with mapping details-->
<servlet>
<servlet-name>CXFServlet</servlet-name>
<servlet-class>org.apache.cxf.transport.servlet.CXFServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
<servlet-name>CXFServlet</servlet-name>
<url-pattern>/services/*</url-pattern>
</servlet-mapping>

</web-app>

In case still the report is not showing then ensure the jdbc connections are prefixed with "jdbc/<Connection Name>" format.

In case for Spring 5, JDK 8, Tomcat 9.x upgrade to javamelody-core-1.99.3.jar

Useful links for Machine Learning in Java

Topic modelling using Mallet:

https://jentery.github.io/507/mallet.html

https://programminghistorian.org/en/lessons/topic-modeling-and-mallet

Mallet Output Visual Interpretation in Excel Macro:

https://wp.nyu.edu/exceltextanalysis/visualize-mallet-topics/

Sentiment Analysis Tool:

Stanford CoreNLP:

https://stanfordnlp.github.io/CoreNLP/tutorials.html

https://www.toptal.com/java/email-sentiment-analysis-bot

https://blog.openshift.com/day-20-stanford-corenlp-performing-sentiment-analysis-of-twitter-using-java/


Vader:

https://github.com/apanimesh061/VaderSentimentJava

Maven dependency for Vader:

<dependency>

<groupId>com.github.apanimesh061</groupId>

<artifactId>vader-sentiment-analyzer</artifactId>

<version>1.0</version>

</dependency>


<!-- https://mvnrepository.com/artifact/log4j/log4j -->

<dependency>

<groupId>log4j</groupId>

<artifactId>log4j</artifactId>

<version>1.2.17</version>

</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.lucene/lucene-analyzers-common -->

<dependency>

<groupId>org.apache.lucene</groupId>

<artifactId>lucene-analyzers-common</artifactId>

<version>8.2.0</version>

</dependency>

Java 8 Heap Memory Issue

Recently I have migrated one of my application from JDK 6 to JDK 8. Once I have migrated to Java 8, I observed the Heap memory is completely getting saturated & CPU consumption is also very high and application is becoming unresponsive.

From thread dump it becomes clear JAXB was taking the memory. Below approach was taken to resolve the issue.

1. Limit the Metaspace max size:
As Metaspace in Java 8 has no limit  hence it was taking the complete heap memory over a period of time; hence set the metaspace max limit using below one in JVM Argument
-XX:MaxMetaspaceSize=512m  - sets the maximum size of the Metaspace to 512 MB
2. JAXB configuration optimization: 
As my application uses lots of XML marshalling & unmarshalling. Hence below addition configuration was required in JVM Argument
-Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true

Useful link for MetaSpace:

http://java-latte.blogspot.com/2014/03/metaspace-in-java-8.html

https://blog.sokolenko.me/2014/11/javavm-options-production.html

Useful link with solutions for OWASP to 10 security issue

http://cwe.mitre.org/data/definitions/1026.html

SqlMap tutorial for Sql Injection checking:

https://nitstorm.github.io/blog/sqlmap-intro/

UTF-8 encoding issue in Response in Tomcat

I have observed UTF-8 encoding issue for JSON response in Tomcat. By default tomcat uses ISO-8859-1. Below are the solution approaches:

Tomcat response (response is appended with ISO-8859-1 charset by Tomcat)

Content-Type: application/json;charset=ISO-8859-1

Solution:

Approach #1:

Add the below code in custom filter or servlet before sending the response

response.setCharacterEncoding("UTF-8");

Approach #2: (Better approach)

Use filter provided by Spring framework as mentioned below; which make the response to UTF-8 (Can add any other charset also).

Please add the below part in web.xml. The respective jar exists in spring-web dependency module.

Snippet to add in web.xml:

<filter>

    <filter-name>encodingFilter</filter-name>

    <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>

    <init-param>

        <param-name>encoding</param-name>

        <param-value>UTF-8</param-value>

    </init-param>

    <init-param>

        <param-name>forceEncoding</param-name>

        <param-value>true</param-value>

    </init-param>

</filter>

<filter-mapping>

    <filter-name>encodingFilter</filter-name>

    <url-pattern>/*</url-pattern>

</filter-mapping>

Externalize of properties file in Tomcat

Below is the approach that can be used for externalize ApplicationResource properties file in Tomcat 7 & 9.

Steps:

1.       The change needs to be done in <tomcat_installation_path>/conf/Catalina/localhost/<APP_NAME>.xml (where data sources are defined)

2.       E.g. ApplicationResource.properties file is kept in D:/AppProperties/TestApp path

3.       For Tomcat 7.x , need to add the folder in classpath by using below tag under <Context> tag

a.       <Resources className="org.apache.naming.resources.VirtualDirContext"

               extraResourcePaths="/WEB-INF/classes=D:/AppProperties/TestApp"/>

4.       For Tomcat 8.x/9.x, you can provide the properties file instead of directory itself, by using below tag under <Context> tag

a.       <Resources>
    <PreResources className="org.apache.catalina.webresources.FileResourceSet"
            base="D:/AppProperties/TestApp/ApplicationResource.properties"
            webAppMount="/WEB-INF/classes/ApplicationResource.properties" />
     </Resources>
b. Alternatively, to configure directory the below one can be used

<Resources>
<PreResources className="org.apache.catalina.webresources.DirResourceSet"
base="D:/AppProperties/TestApp"
webAppMount="/WEB-INF/classes"/>

</Resources>

5.       This will load the properties file from the external location; hence remove the properties file from WEB-INF/classes

How to do a full-text search in SVN repository

SVN clients like TortoiseSVN does not come with the content search support. It can be done by using Git SCM client.

Installation:

Please follow the below steps on Windows

Goto https://git-scm.com/download/win

Download & install the exe file

Copying SVN Content to Local Drive:
Create a folder in any drive. e.g. D:\myrepo

Go to windows command prompt

Then from command prompt navigate to D:\myrepo (using cd)

Clone the SVN repository to the local drive (D:\myrepo) by executing below command

git svn clone <SVN_URL>

You will be prompted for username/password in command prompt while downloading the SVN content

Search the content:

Once the download is finished in D:\myrepo; execute the below command providing the text to be searched in <keyword>.

git grep -i <keyword> [e.g. git grep -i abcd]

-i is used to provide case-insensitive content search. To make the search case-sensitive remove -i option


The list of files with contents will be printed in the command prompt

Google AI tutorial/crash courses

Google tutorial/crash courses for learning AI:
https://ai.google/education/#?modal_active=none

Useful Links for JEE Devlopment

1. Links on Burlap Web service creation. A very easy way for Java to Java remoting:

http://www.devx.com/java/Article/27300/0/page/1

http://www.christianschenk.org/blog/webservices-with-hessian-and-burlap/




2. J2EE application Deployment Problem:

Problem:

java.lang.IllegalStateException: Web app root system property already set to

a different value: 'webapp.root'

Resolution

http://forum.springsource.org/archive/index.php/t-32873.html

http://forum.springsource.org/archive/index.php/t-24073.html


3. Solution on Axis 2 issue on Upgrade to 1.7.4 from 1.4.1:

Configuration required:

<parameter name="disableREST" locked="false">true</parameter>
Detail explanation in below blog

http://alloutfornoloss.com/axis2-epr-issue/

4.REST API Naming convention:
https://google.github.io/styleguide/jsoncstyleguide.xml?showone=Property_Name_Format#Property_Name_Format

Printing the DBMS_OUTPUT.PUT_LINE output from oracle to System.out in Java

Often we need to debug a Oracle stored procedure which is called from Java. In that case, it is helpful to log the DBMS_OUTPUT.put_line from Java using JDBC driver.
The below link guides a way to retrieve DBMS_OUTPUT.put_line from JDBC:

https://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:45027262935845

Eclipse Papyrus: An easy way to create UML diagrams

 As a software designer/architect one would always need to take the help of UML. There are many open source tools available in market to create UML diagrams.

Visual Paradigm,Star UML to name a few. But my best personal choice is Eclipse Papyrus. This tool is very easy to learn to create UML diagrams.


Check out youtube videos on how to create various UML diagrams using Eclipse Papyrus:
https://www.youtube.com/playlist?list=PLoWne5q-c9E_Q2_eAUZKPDA5K0V-O5zXs


Check out Eclipse Papyrus here


https://www.eclipse.org/papyrus/


Download Link

https://www.eclipse.org/papyrus/download.html

Useful Book Links

Books on programming:
https://github.com/EbookFoundation/free-programming-books/blob/master/free-programming-books.md
https://github.com/bjut-hz/E-Books
https://github.com/sivaprak/myBooks
https://github.com/sanigo/books
O'Reilly Free Books:
https://gist.github.com/viruscmd/5960adf1e5b5cf1e3d465dd3ad9e1aa8
https://gist.github.com/abstractart/7b969dd9004cd11e04ba0a031fa4ab6c
Togaf Books
https://github.com/pankajchopra/togaf
Blockchain Books:
https://github.com/ShubhamTatvamasi/blockchainbooks
Books on Hadoop/Big Data
https://github.com/TechBookHunter/Free-Hadoop-Books/tree/master/book

Axis 2 Directory traversal vulnerability

Axis 2 Directory traversal  security vulnerability

Recently I have encountered one security issue of Axis 2 (1.4.1) service. The attacker can navigate to the axis.xml using the link https://victim.com/axis2/services/Version?xsd=../conf/axis2.xml & can see the Axis 2 username & password. Then attacker can deploy any malicious service to hack the system.

The issue seems to happen if the Axis 2 version <1.5.3. Upgrading the existing version to 1.5.3 (at minimal, upper versions also support) solves the problem.

The root cause of the issue is below configuration in Axis 2 1.4.1 version:

<transportReceiver name="http"

                       class="org.apache.axis2.transport.http.SimpleHTTPServer">

        <parameter name="port">8080</parameter>

SimpleHTTPServer does not block any request & hence directory traversal is possible.

I have followed the below steps to upgrade the Axis 2 from 1.4.1 to 1.5.3

1.     Upgrade the Axis 2 version to 1.5.3. & update the jars

2.     Once the JARS have been upgraded, change the below ones in conf\axis2.xml

replace

<transportReceiver name="http"

                       class="org.apache.axis2.transport.http.SimpleHTTPServer">

        <parameter name="port">8080</parameter>

with below one

<transportReceiver name="http"

                       class="org.apache.axis2.transport.http.AxisServletListener">

        <parameter name="port">8080</parameter>

    </transportReceiver>

    <transportReceiver name="https"

                       class="org.apache.axis2.transport.http.AxisServletListener">

        <parameter name="port">8443</parameter>

    </transportReceiver>

3.     Comment TCPTransportSender in axis2.xml

<!--

    <transportSender name="tcp"

                     class="org.apache.axis2.transport.tcp.TCPTransportSender"/>-->