Monday, May 8, 2023

Externalization of Logs from Docker Container

For Docker , if we want to persist any data after the container is removed we can use the following command format:

docker run -d -p 9090:8080 -v <OS File Path>:<Docker container Path> tomcat-test-webapp

Here the logs with be stored in OS file/folder path instead of storing in  Docker container path

docker build -t tomcat-test-webapp .

docker run -d -p 9090:8080 -v F:/docker_data/logs:/usr/local/tomcat/logs tomcat-test-webapp


The -v option is used to create volume which persists in OS even after the container is removed.


With container name & Catalina Options it should look like below:

docker run -d --name testDockerWebApp -p 9090:8080 -v F:/docker_data/logs:/usr/local/tomcat/logs -e CATALINA_OPTS="-Xms512M -Xmx512M" tomcat-test-webapp

Format with log, connection pool, properties file externalization:

docker run -d --name <image_name> -p <external_port>:<Docker Internal Port> -v /opt/app_log/<app_name>:/usr/local/tomcat/logs -v /opt/app_cp/<app_name>:/usr/local/tomcat/conf/Catalina/localhost -v /opt/app_prop/<app_name>:/usr/local/properties/<app_name> -e CATALINA_OPTS="-Xms512M -Xmx512M" <docker_hub_image_name>

Friday, March 24, 2023

Deploy WAR file in Docker Container

Pre-requisite: Docker Desktop for Windows to be installed & started

Steps:

Create a WAR file [e.g. TestApp.war]

Create a Dockerfile

Place both of them (WAR & Dockerfile) in same folder (e.g. D:\DevOps)

Navigate to that folder (D:\DevOps) & open command prompt

Run below command to create a image name. Here tomcat-test-webapp is the name of image 

docker build -t tomcat-test-webapp .

To run the image use below command

docker run -d -p 8080:8080 tomcat-test-webapp

In case the port to be different use the below structure

docker run -d -p <custom port>:8080 tomcat-test-webapp

Content of Dockerfile

FROM tomcat:9.0.52-jdk8-corretto

COPY ./TestWebApp.war /usr/local/tomcat/webapps

EXPOSE 8080

CMD ["/usr/local/tomcat/bin/catalina.sh","run"]


https://www.youtube.com/watch?v=B9vy3DMHo2I

Pushing Image to Docker Hub

https://www.cloudbees.com/blog/using-docker-push-to-publish-images-to-dockerhub


Pulling Images from Docker Hub & Run

docker login

docker pull <docker_user_name>/tomcat-test-webapp:latest

docker run -d -p 9080:8080 <docker_user_name>/tomcat-test-webapp

In case to run image with memory settings or other Catalin options ; use below command:

docker run -d -p 9080:8080 -e CATALINA_OPTS="-Xms512M -Xmx512M"  <docker_user_name>/tomcat-test-webapp


Auto Deploy the changes from Docker Hub:

WATCHTOWER_POLL_INTERVAL is set in sec.


docker run -d --name watchtower -e REPO_USER=<> -e REPO_PASS=<> -e WATCHTOWER_POLL_INTERVAL=30 -e WATCHTOWER_CLEANUP=true -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower <container_name>


https://alexgallacher.com/auto-update-docker-containers-by-setting-up-watchtower/

https://containrrr.dev/watchtower/

https://www.geekyhacker.com/how-to-use-spotify-docker-maven-plugin/


Wednesday, January 25, 2023

Software Links

 1. Mockoon https://mockoon.com/

2. PlanetUML

3. Junit Auto test case generator (https://www.diffblue.com/community-edition/download/)

4. https://blog.jetbrains.com/idea/2024/07/top-tools-for-java-developers-in-2024/

5.https://www.thoughtworks.com/en-in/insights/blog

Thursday, January 12, 2023

Opensearch Install Windows

OpenSearch is a ElasticSearch fork from Amazon

Download Opensearch from https://opensearch.org/downloads.html

Extract opensearch-2.4.1-windows-x64.zip in a folder in Windows

Openserach comes with JDK 17

Set JAVA_HOME for JDK 17 in <OpenSearch Extracted Folder>/opensearch-2.4.1/bin/opensearch-env.bat

set JAVA_HOME=<OpenSearch Extracted Folder>/opensearch-2.4.1/jdk

Open <OpenSearch Extracted Folder>/opensearch-2.4.1/config opensearch.yml

Add the below line to disable secure connection 

plugins.security.disabled: true

Now go to <OpenSearch Extracted Folder>/opensearch-2.4.1/opensearch-windows-install.bat

from command prompt.

Once started , navigate to http://localhost:9200/



Monday, November 28, 2022

Running a continuos job using Reactor

In this section we will try to execute a job at a certain interval, using Spring Reactor framework. The job should be defined in the subscribe method.

Maven Dependency:

<dependency>

    <groupId>io.projectreactor</groupId>

    <artifactId>reactor-core</artifactId>

    <version>3.4.24</version>

</dependency> 

Code Snippet:

Running a job at a certain interval (10 sec) forever.

Mono.just("SD").delaySubscription(Duration.ofSeconds(10)).repeat(()->true).subscribe(a->System.out.println(a));

Wednesday, September 14, 2022

Installing Kotlin on Eclipse

Recently I have faced problem to install Kotlin Plugin for Eclipse IDE. As per the StackOverflow post  the plugin has been removed. The nelwy forked linked that can be used to install Kotlin in Eclipse is

https://github.com/bvfalcon/kotlin-eclipse-2022


Sunday, April 17, 2022

AWS Translate using AWS JS SDK

 In this post, I will discuss how to use AWS Javascript SDK V3 for AWS Translate.

Prerequisite: Node.js should be pre-installed

Steps:

  • Create a folder e.g. AWSTranslateApp
  • From command prompt navigate to folder "AWSTranslateApp"
  • Type npm init
  • This will create Node project
  • Install AWS JS SDK by typing "npm install @aws-sdk/client-translate" under folder "AWSTranslateApp"
  • Now open the "AWSTranslateApp" folder in VS Code
  • Create a file translateExample.js

Below is the code snippet for translateExample.js

const { TranslateClient, TranslateTextCommand } = require("@aws-sdk/client-translate");

 async function getTranslatedData(){

  const client = new TranslateClient({ region: "<region_name>", 

  credentials: {

    accessKeyId: '<access_key>', 

    secretAccessKey: '<secret_key>'

  } 

});

  const params = {

    SourceLanguageCode: "en",

    TargetLanguageCode: "es",

    Text: "Hello, world"

  };

      const command = new TranslateTextCommand(params);

      const data = await client.send(command);   

      const jsonData =await data.TranslatedText;

      console.log(jsonData);

  }

 getTranslatedData();


  • Replace the region, access key, secret key with the actual values. In my case the region was 'us-east-1'.
  • Once done, you can now run the program using below command from command prompt

        node translateExample

References:

https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-translate/index.html



Saturday, April 9, 2022

AWS Service Call from Java SDK

 Here in this post I will discuss how to use AWS Translate Service from Java Code.

Steps:

1. Add the below dependency in pom.xml

<dependency>

    <groupId>com.amazonaws</groupId>

    <artifactId>aws-java-sdk-translate</artifactId>

    <version>1.12.194</version>

</dependency>

2. Add below imports in Java File

import com.amazonaws.auth.AWSStaticCredentialsProvider;

import com.amazonaws.auth.BasicAWSCredentials;

import com.amazonaws.regions.Regions;

import com.amazonaws.services.translate.AmazonTranslate;

import com.amazonaws.services.translate.AmazonTranslateClient;

import com.amazonaws.services.translate.model.TranslateTextRequest;

import com.amazonaws.services.translate.model.TranslateTextResult;

3. Below is the code snippet; provide the accessKey & secretKey to access the service

BasicAWSCredentials credentials = new BasicAWSCredentials("<access_key>",
"<secret_key>");
AmazonTranslate translate = AmazonTranslateClient.builder()
.withCredentials(new AWSStaticCredentialsProvider(credentials)).withRegion(Regions.US_EAST_1).build();
TranslateTextRequest request = new TranslateTextRequest().withText("Hello, world").withSourceLanguageCode("en")
.withTargetLanguageCode("es");
TranslateTextResult result = translate.translateText(request);
System.out.println(result.getTranslatedText());

AWS Service Call from Postman

Many of the times you need to use AWS services via Postman for testing.

I am using Amazon Translate Service, below are the configuration made to call the service:

In Postman choose the Post method for call

Service URL: https://translate.us-east-1.amazonaws.com/

In "Authorization" tab choose "Type" "AWS Signature"

Provide AccessKey, SecretKey, AWS Region, Service Name

The region I am using is "us-east-1" , Service Name will be "translate"

Then Move to "Headers" tab & add the following Headers

Content-Type: application/x-amz-json-1.1

X-Amz-Target: AWSShineFrontendService_20170701.TranslateText

N.B. X-Amz-Date will be generated by Postman automatically

Under "Body", select "raw", and added the following sample body:

{
    "SourceLanguageCode": "en",
    "TargetLanguageCode": "es",
    "Text": "Hello, world"
}

Now hit the Send button & check the result.

Clicking on the "Code" in Postman you can also get the cod for Java/Node JS and many other languages.

Helpful Links:

https://stackoverflow.com/questions/59128739/how-to-use-aws-translate-translatetext-api-via-postman

https://docs.aws.amazon.com/translate/latest/dg/API_Reference.html

Wednesday, March 30, 2022

Maven Repository configuration in pom.xml

Many a times you might face to downlod maven dependency in Jenkins due to netwok issue. In that case, Maen repository needs to be configured in pom.xml.

Below is the snippet you can use to configure maven repo.

<project>

..................

    <repositories>

<repository>

<id>central</id>

<name>Central Repository</name>

<url>https://repo.maven.apache.org/maven2</url>

<layout>default</layout>

<snapshots>

<enabled>false</enabled>

</snapshots>

</repository>

</repositories>


<pluginRepositories>

<pluginRepository>

<id>central</id>

<name>Central Repository</name>

<url>https://repo.maven.apache.org/maven2</url>

<layout>default</layout>

<snapshots>

<enabled>false</enabled>

</snapshots>

<releases>

<updatePolicy>never</updatePolicy>

</releases>

</pluginRepository>

</pluginRepositories>

    

</project>

Thursday, March 10, 2022

Calling external services using RestTemplate

 Though today's date, microservice with  Spring boot is a very popular architecture; but along with new architectural apps we need to maintain legacy apps also.

In legacy apps, we will often found external http calls. In my case, most of the times I found the legacy code is using Apache commons httpclient or plain java http calls.

One of the common mistake is, after making the call we often leave the http connection open instead of closing. This causes probelm when the called service is down & in those cases the calling apps will have memory issue.

To solve this, we can  use Spring RestTemplate instead of  other http clients. The main reason is, the code is tiny & crisp ; also the closing of connection is also handled by Spring.

Steps:

1. Add below maven dependency:


        <dependency>

            <groupId>org.springframework</groupId>

            <artifactId>spring-web</artifactId>

            <version>3.0.6.RELEASE</version>

        </dependency>

2. Below is a sample code: Assumption is Code is sending JSON request & receiving JSON response


public static String callRestService(String serviceUrl, String strRQ) {

SimpleClientHttpRequestFactory clientHttpRequestFactory= new SimpleClientHttpRequestFactory();

//Connect timeout in milisec.

clientHttpRequestFactory.setConnectTimeout(1000);


//Read timeout in milisec

clientHttpRequestFactory.setReadTimeout(1000);

RestTemplate restTemplate = new RestTemplate(clientHttpRequestFactory);

HttpHeaders headers = new HttpHeaders();

headers.setAccept(Arrays.asList(MediaType.APPLICATION_JSON));

headers.setContentType(MediaType.APPLICATION_JSON);

HttpEntity<String> entity = new HttpEntity<String>(strRQ, headers);

ResponseEntity<String> response=restTemplate.exchange(serviceUrl, HttpMethod.POST, entity, String.class);

return response.getBody();

}


Tuesday, August 24, 2021

TechBlog Links

 As a good software engineer, one needs to keep updated on the emerging technologies & the use cases where to apply those technologies. 

To gain more knowledge, one should follow the technology blogs; below are few of my favorite technology blogs:

1. https://blog.allegro.tech/

2. https://doordash.engineering/

3. https://engineering.cerner.com/

4. https://booking.design/

5. https://netflixtechblog.com/

6. https://medium.com/expedia-group-tech

7. https://comcast.github.io/blog.html

8. https://shekhargulati.com/

9. https://www.appsdeveloperblog.com/keycloak-rest-api-create-a-new-user/



Saturday, August 14, 2021

Routing Http Calls through Proxy

Proxy server is one of the network backbone for any corporate network. There are 2 types of proxy setup

1. Forward Proxy: Used for the outbound traffic going from your network to Internet. It is also called Client Side Proxy.

2. Reverse Proxy: Used for inbound calls where traffic is coming from Internet to your network.

Below picture depicts the 2 proxies


In this article, we will discuss on Forward Proxy setup & dicuss how to route the calls through Forward Proxy from Java Http client calling codes.

Step 1: Forward Proxy Setup in Windows 

There are many open source Forward Proxy available like Apache Httpd , Squid etc.

I have chosen Squid for Proxy setup as it has a very easy setup.

First download Squid from https://squid.diladele.com/ & install the msi

This will be installed as a Windows service.

Step 2: Post Installation configuration of Squid

Once installed you will find Squid tray

Click "Open Squid Configuration" option

Add the below one at last of the configuration file, this will speed up the traffic calls.

dns_v4_first on

Step 3: Client calls from Java routing through Proxy

Suppose you want to call https://www.google.com/ from Java Client.

In this example, we will use Spring RestTemplate.

Create a new Maven project.

3.1. Add the below dependencies

<dependency>

<groupId>org.apache.httpcomponents</groupId>

<artifactId>httpcore</artifactId>

<version>4.4.13</version>

</dependency>

<!-- https://mvnrepository.com/artifact/org.springframework/spring-web -->

<dependency>

    <groupId>org.springframework</groupId>

    <artifactId>spring-web</artifactId>

    <version>4.3.30.RELEASE</version>

</dependency>


3.2. Sample Code for Proxy call:

import java.net.InetSocketAddress;

import java.net.Proxy;

import java.net.Proxy.Type;

import org.springframework.http.ResponseEntity;

import org.springframework.http.client.SimpleClientHttpRequestFactory;

import org.springframework.web.client.RestTemplate;

public class ProxyHttpClient {

private static String PROXY_SERVER_HOST = "localhost";

private static int PROXY_SERVER_PORT = 3128;

public static void main(String[] args) {

Proxy proxy = new Proxy(Type.HTTP, new InetSocketAddress(PROXY_SERVER_HOST, PROXY_SERVER_PORT));

SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();

requestFactory.setProxy(proxy);

RestTemplate restTemplate = new RestTemplate(requestFactory);

ResponseEntity<String> responseEntity = restTemplate.getForEntity("https://www.google.com/", String.class);

String bodyStr = responseEntity.getBody();

System.out.println("bodyStr:" + bodyStr);

}

}

Links:

Squid Setup

Proxy Concept

Wednesday, June 9, 2021

Event Handling in Spring

Suppose you are working on a Order Management system. Once the order is placed, the  system needs to do the following taks:

1. Send email notification to customer

2. Send request to Payment processing system to make payment.

Generally, in traditional way of programming, once the order is placed we call below 2 methods:

sendEmailToCustomer()

makePayment()

Now suppose , the product owner gives you a requirement to send email notification to seller also once the order is placed. To do that, now you need to introduce another method, sendEmailToSeller, along with the above 2 methods.

This approach has a drawback. If the order is created from multiple places, we need to introduce this change in all these places.

We can handle the same problem in event driven approch. We can consider Order creation as an event; hence it becomes a producer for event & sending email to cutomer , making payment & sending email to Seller become the event consumers.

Spring framework comes with an in-built support for Event Driven processing. It requires 3 elemts for an event:

1. the Event itself

2. Pulisher of the event

3. Consumer/Listener of the event

All of these are handled in Spring framework in an elegant way. 

Prerequisite: 

Java 8

Spring framework version: 4.3.30.RELEASE

Maven dependency:

                <dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-core</artifactId>

<version>4.3.30.RELEASE</version>

</dependency>

<dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-context</artifactId>

<version>4.3.30.RELEASE</version>

</dependency>

Event: The event can be any Java Bean model class; for brevity have removed the getters & setters. You can add @Getter & @Setter annotation from lombok library also.

public class OrderEvent {

private String itemName;

private int quantity;

}

Event Publisher: Spring comes with an in built ApplicationEventPublisher class defined in org.springframework.context.ApplicationEventPublisher.

You can publish the event like below: 

@Service

public class OrderEventProducer {

@Autowired

private ApplicationEventPublisher publisher;

public void publishTestEvent() {

OrderEvent order = new OrderEvent();

order.setItemName("Pen");

order.setQuantity(5);

System.out.println("Puslishing order");

publisher.publishEvent(order);

}

}

Event Listener: Once the event is pusblished, it can be consumed. The consumers are called EventListner. Spring comes with below features for Event listener/consumers
1. The consumer can be asynchronous , add @EnableAsync annotation at class level & the method to be processed async. way need to add @Async annotation.
2. For multiple consumers orders can be set with @Order annotation
3. Any method can be marked as event listener with  @EventListener annotation
4. The event listener method must have the same event argument in consumer method as published from ApplicationEventPublisher.

The code snippet will look like below.

@Component
@EnableAsync
public class OrderEventListener {
@Async
@EventListener
@Order(1)
public void sendEmailToCustomer(OrderEvent event) {
System.out.println("Starting email sending");
delay();
System.out.println("sendEmail:" + event);
}

@EventListener
@Order(2)
public void makePayment(OrderEvent event) {
System.out.println("makePayment:" + event);
}

private void delay() {

try {
TimeUnit.SECONDS.sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

Now, you can easily add other consumer methods for same event with OrderEvent  as parameter. No code change required at producer end.

Code Example:


Furthur Reading:


Tuesday, June 8, 2021

Caching using Hazelcast

Caching is one of the important aspect when we do system design as it enhance the performance.

In many of my applications I have used Ehcache as a cache provider with Spring applications. One of the problem with Ehcache is ; the cache resides in Single Node. 

Let's consider the below scenario:

Suppoe, we have an application where we have a method which provides Book details based on isbn provided in input. The method to findBookByIsbn is costly & cache is implemented.

Now, we call findBookByIsbn for isbn 1 & through Load Balancer, it goes to Node 1 & it fetches the data from DataBase & store in cache.

Now, another call is made to findBookByIsbn for isbn 1 & through Load Balancer, it now goes to Node 2. In this case it again fetches the data from DataBase & store in cache.

Hence, for same data (isbn=1) the db call is again made in DataBase as the cache resides in each node seperately. 

The architecture is deplicted in below image



Now, you can solve this problem by creating an Embedded Distributed Cache (aka Replicated Cache). In this case, Cache of Node 1 interacts with Node 2 & replicates the data. The architecture will look like below:





This technique can be implemented using Ehcache with JGroups.


EHCache Replicated Cache Tutorial Links:


As in Ehcache-Jgroups combination, we need to do lot of manual configuration, another good alternative is using Hazelcast. In this note, I am going to give you the steps you required to use HazelCast as cache manager

Pre-requisite:

Requied Java Version: 8

Spring Framework Verion used: 4.3.30.RELEASE

The Cache data type should implement Serializable interface

Step #1: Adding Maven dependency for Hazelcast Spring integration & Spring Context upport

                <dependency>

<groupId>com.hazelcast</groupId>

<artifactId>hazelcast-spring</artifactId>

<version>4.2</version>

</dependency>

        <dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-context-support</artifactId>

<version>4.3.30.RELEASE</version>

</dependency>

 Step #2: Defing the method. The method must be defined in a Spring Bean class (Class having annotation Service/Component or defined in XML)

@Cacheable("bookIsbnCache")

public Book findBookByIsbn(String isbn) {

        // DB / Service call goes here 

        }

Step #3: Define the cache in application context xml

<beans xmlns="http://www.springframework.org/schema/beans"

xmlns:context="http://www.springframework.org/schema/context"

xmlns:p="http://www.springframework.org/schema/p"

xmlns:cache="http://www.springframework.org/schema/cache"

xmlns:hz="http://www.hazelcast.com/schema/spring"

xmlns:mvc="http://www.springframework.org/schema/mvc"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="

        http://www.springframework.org/schema/beans     

        http://www.springframework.org/schema/beans/spring-beans.xsd

        http://www.springframework.org/schema/context 

        http://www.springframework.org/schema/context/spring-context.xsd

        http://www.springframework.org/schema/mvc

        http://www.springframework.org/schema/mvc/spring-mvc.xsd

       http://www.springframework.org/schema/cache http://www.springframework.org/schema/cache/spring-cache.xsd

       http://www.hazelcast.com/schema/spring

       http://www.hazelcast.com/schema/spring/hazelcast-spring.xsd">

<!-- Other bean definition-->

<cache:annotation-driven
cache-manager="cacheManager" />

<hz:hazelcast id="instance">
<hz:config>

<hz:cluster-name>TestHzCluster</hz:cluster-name>

<!--  used for clustering.
<hz:network port="5701" port-auto-increment="false">
<hz:join>
<hz:multicast enabled="false" />
<hz:tcp-ip enabled="true">
<hz:members>x.x.x.x, y.y.y.y</hz:members>
</hz:tcp-ip>
</hz:join>
</hz:network>
-->

<hz:map name="bookIsbnCache" time-to-live-seconds="60"
in-memory-format="BINARY">
<hz:eviction eviction-policy="LRU"
max-size-policy="PER_NODE" size="100" />
</hz:map>
</hz:config>
</hz:hazelcast>

<bean id="cacheManager"
class="com.hazelcast.spring.cache.HazelcastCacheManager">
<constructor-arg ref="instance" />
</bean>

</beans>

That' it. You can deploy your code in different ports in localhost & you will be able to see the cache is replicated among differnt nodes.

Below is the link for working demo:

https://github.com/souravdalal/SpringHazelcastCacheDemo

Furthur Reading:

Cache Topologies:




Hazelcast with Spring Boot:



Sunday, May 30, 2021

Migrating Java 6/8 projects to JDK 11

Recently, I have migrated some of my applications from JDK 6 & 8 to JDK 11. As Oracle JDK 11 has a license cost associated with it; I have used Amazon Corretto JDK 11; which is a no-cost, multiplatform, production-ready distribution of OpenJDK.

Below are the steps followed for JDK 11 upgrade

Step 1: Install JDK 11 & Maven 3.6.3 & set the path

You can check the https://mkyong.com/java/how-to-set-java_home-on-windows-10/ for details of Java path settings

You can check the   https://mkyong.com/maven/how-to-install-maven-in-windows/ for detail of Maven path settings

Step 2: Next ,you need to do couple of changes in your application pom.xml:

If you have Maven compiler version set for JDK 6 or 8 in any of the below format; then remove those 

Format 1:

<properties>

    <maven.compiler.target>1.8</maven.compiler.target>

    <maven.compiler.source>1.8</maven.compiler.source>

</properties>


Format 2:

<plugins>

    <plugin>    

        <artifactId>maven-compiler-plugin</artifactId>

        <configuration>

            <source>1.8</source>

            <target>1.8</target>

        </configuration>

    </plugin>

</plugins>


We need to add the below compilation setting under <build> tag & surefire plugin settings should be modified as below:

  <plugins>

<plugin>

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-surefire-plugin</artifactId>

<version>2.22.0</version>

<configuration>

<argLine>

--illegal-access=permit

</argLine>

</configuration>

</plugin>

<plugin>

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-compiler-plugin</artifactId>

<version>3.8.0</version>

<configuration>

<release>11</release>

</configuration>

</plugin>

</plugins>

Step 3: If your application is using Spring Framework 3.x or 4.x then you need to upgrade the Spring version to atleast 5.1.0.RELEASE. 

As 5.1.0.RELEASE is the compatible version with JDK 11. Otherwise, you will not get compile time exception but at runtime you will get exception like below:

Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: Failed to read candidate component class: file [MyClass.class]; nested exception is java.lang.ArrayIndexOutOfBoundsException: 11315

You can find more details in https://www.javagists.com/beandefinitionstoreexception-failed-to-read-candidate-component-class

Upgrading Spring version can give you comile time time error based on methods which has been removed from upper Spring version. E.g. Spring 3.x have methods in JDBCTemplate for queryForInt, queryForLong; which has been deprecated &  queryForObject is introduced from Spring 4.x

Also, if you are using Spring JDK Timer (org.springframework.scheduling.timer); then it needs to be upgraded to  Spring Quartz Scheduler.

Sunday, April 11, 2021

Viewing Tomcat logs on Web Browser

While doing the development, many times developer requires to check the logs of tomcat as well as application logs in Development/QA environments. 
Hence, to check the logs the developer needs to connect to the remote environment to check the log files physically. 

Wouldnot it be nicer if we can view the logs from in browser itself.

Tomcat provides a nice way to handle this. You can view the logs from browser itself. Below are the steps.

Steps:
1. Download Tomcat 9.x & extract the zip.
2. Move to <TOMCAT_INSTALL_DIR>/conf/Catalina/localhost
3. Create a file name logs.xml
4. Add the below line & save the logs.xml file. 
<Context override="true" docBase="${catalina.base}/logs" path="/logs" />
5. Additionally, navigate to web.xml under <TOMCAT_INSTALL_DIR>/conf
6. Change the value of  "listings" parameter to true declared under DefaultServlet as below:

<servlet>
        <servlet-name>default</servlet-name>
        <servlet-class>org.apache.catalina.servlets.DefaultServlet</servlet-class>
        <init-param>
            <param-name>debug</param-name>
            <param-value>0</param-value>
        </init-param>
        <init-param>
            <param-name>listings</param-name>
            <param-value>true</param-value>
        </init-param>
        <load-on-startup>1</load-on-startup>
    </servlet>

7. This enables to view the files under the logs folder.
8. Now Start the tomcat
9. Type in browser  http://localhost:8080/logs/

You will see a output like below , providing the logs under  <TOMCAT_INSTALL_DIR>/logs folder



Some times, the logger logs are generated outside of Tomcat logs folder. 
Suppose, your application logs are generated in under D:/logs/<APP_NAME> folder.

In that case, you can create another file in named <APP_NAME>.xml under <TOMCAT_INSTALL_DIR>/conf/Catalina/localhost

In <APP_NAME>.xml you can put the below content

<Context override="true" docBase="D:/logs/<APP_NAME>" path="/<APP_NAME>" />

Now navigating to http://localhost:8080/<APP_NAME>/ will show you the logs generated under D:/logs/<APP_NAME> folder.


Tuesday, March 2, 2021

Handling Retry with Spring-Retry

 Suppose you have a method for which you want to retry at a specific interval if any exception occurs.

Below are the steps using Spring Retry:

Add below maven dependenciess:

<dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-aop</artifactId>

<version>3.0.5.RELEASE</version>

</dependency>

<dependency>

<groupId>org.aspectj</groupId>

<artifactId>aspectjweaver</artifactId>

<version>1.6.11</version>

</dependency>


<dependency>

<groupId>org.springframework.retry</groupId>

<artifactId>spring-retry</artifactId>

<version>1.1.2.RELEASE</version>

</dependency>

Steps for call:

                SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();

retryPolicy.setMaxAttempts(5); //Max retry # 


FixedBackOffPolicy backOffPolicy = new FixedBackOffPolicy();

backOffPolicy.setBackOffPeriod(1500); // Retry @ every1.5 seconds


RetryTemplate template = new RetryTemplate();

template.setRetryPolicy(retryPolicy);

template.setBackOffPolicy(backOffPolicy);


return template.execute(new RetryCallback<String, Exception>() {


public String doWithRetry(RetryContext context) throws Exception {

                            // Asumption method m1 is sending String as return type

 String outputStr =m1();

return outputStr;

}

});


RetryCallback should be defined with <[ReturnType Of Method],[Exception to be Retried]>.

Here in case, I have assumed the called method is returning String & retry in case of an instance of Exception is thrown.

Below are the classes need to import:

import org.springframework.retry.RetryCallback;

import org.springframework.retry.RetryContext;

import org.springframework.retry.backoff.FixedBackOffPolicy;

import org.springframework.retry.policy.SimpleRetryPolicy;

import org.springframework.retry.support.RetryTemplate;


Saturday, December 19, 2020

Secured Web Service testing with Burp Suite

After you have intercepted the request in Burp Suite, follow below steps to send request for secured web services

Sending Request to Repeater:

Right click under "Proxy"-->"Intercept"-->"Raw" tab

Click on option "Send to Repeater"

Goto Repeater tab. You will now be able to see the same request as intercepted over here also.

If the Web service is secured one; you will  have below details in SOAP header:

1. Username

2. Password in Digest mode (encrypted)

3. Nonce (Should be unique in each request)

4. Created date

For secured web service testing, you need to install WS-Security Extension from Burp Suite App store.

Steps for installation of WS-Security extension:

Goto "Extender" --> "BApp Store"

Navigate to "WS-Security" & install it.

Once installed , the extension will be seen as a new tab named "WS-Security"

Configuration of WS-Security:

Navigate to "WS-Security" tab

Provide the password in Plain text "Password" text box

Now click "Turn WS-Security on".

Configuration of Scope:

Goto "Target"--> "Scope"

Click on "Add" button

Provide the Web Service End Point URL

Configuring WS-Security details in request:

Now we need to configure below 3 details in "Extender" tab, so that the nonce, created date  & password digeest  can be done automatically by WS-Security extension.

Replace the password value in SOAP request with #WS-SecurityPasswordDigest

Replace the value in nonce tag with #WS-SecurityNonce

Replace the value in created tag with #WS-SecurityCreated

This will enable to dynamically change the values with the one configured in "WS-Security" tab while making the SOAP request.

Click on the "Send" button in under "Repeater" tab. 

You will see the reponse in right hand side. 






Web Service Testing with Burp Suite & SOAP UI

Installation of Burp Suite:

Pre-requisite:

Java 8 should be already installed.

Goto below link:

https://portswigger.net/burp/releases?initialTab=community

If you are using Java 8, then download the JAR version for release version 2020.2.Java 8 is no longer supported for version upper than this.

Start Burp Suite:

Goto windows command prompt.

Navigate to the folder where the JAR is downloaded.

Run the below command

java -jar burpsuite_community_v2020.2.1.jar

Intercepting request:

Once the Burp Suite is open, first step is to turn on interceptor.

Navigate to "Proxy" --> "Options" tab

By Default the Interceptor is hosted in 8080 port. This can be changed to port you want.

Check the status is "Running".

Now  Navigate to "Proxy"-->"Intercept" tab

Now turn on the Interceptor by clicking button "Intercept is on".

SOAP UI Proxy Configuration:

Download SOAP UI from below link:

https://www.soapui.org/downloads/soapui/soapui-os-older-versions/

Start SOAP UI.

Goto "Files"-->"Preferences"

Click on "Proxy Settings"

Select "Manual" option & provide Host "127.0.0.1" & Port as "8080" (same as Burp Suite interceptor port)

Now doing any SOAP request in SOAP UI will be intercepted in Burp Suite & will be shown in "Proxy"--> "Intecept"-->"Raw"




Wednesday, December 9, 2020

Concurrency In RxJava


RxJava achieves concurrency through the Schedulers. 

Most commonly used Schedulers are IO & Computation:

Schedulers.io : Used for IO bound tasks (e.g.network call or Database call.)

Schedulers.computation : Used for CPU bound tasks.  (e.g. sorting large array in Java code)

The difference of CPU bound Vs IO bound task:

In case of IO bound task, we can have more theads , than,  the no of CPU cores of running machine. Because CPU is idle when the IO operaion is called.

Whereas in case of CPU bound task, as it is purely computational (e.g. Performing sorting in Java ArrayList); hence we should avoid creating threads more than the no of CPU core in the running machine.


subscribeOn : runs the tasks in new thread (start to end)

observeOn  : threading is applied only on the downstram task. (Operations defined after observeOn call)


Example:

In below example, we have taken a String , then transform the String to uppercase  , then printed the value.


File Name: ObsSubsEx.java

import java.util.concurrent.TimeUnit;


import io.reactivex.Observable;

import io.reactivex.schedulers.Schedulers;

public class ObsSubsEx {


public static void main(String[] args) throws InterruptedException {


Observable.just("subscribeOn One").subscribeOn(Schedulers.computation()).map(ObsSubsEx::toUpper).subscribe(ObsSubsEx::printVal);


TimeUnit.SECONDS.sleep(1);


Observable.just("subscribeOn Two").map(ObsSubsEx::toUpper).subscribeOn(Schedulers.computation()).subscribe(ObsSubsEx::printVal);


TimeUnit.SECONDS.sleep(1);


Observable.just("observeOn").map(ObsSubsEx::toUpper).observeOn(Schedulers.computation()).subscribe(ObsSubsEx::printVal);


TimeUnit.SECONDS.sleep(1);

}

private static String toUpper(String val) {

System.out.println("Uppercase done on thread:"+Thread.currentThread().getName());

return val.toUpperCase();

}


private static void printVal(String val) {

System.out.println("Final value is:"+val+":Thread:"+Thread.currentThread().getName());

}

}


Let's see from output log how the flow works:

Uppercase done on thread:RxComputationThreadPool-1
Final value is:SUBSCRIBEON ONE:Thread:RxComputationThreadPool-1

Uppercase done on thread:RxComputationThreadPool-2
Final value is:SUBSCRIBEON TWO:Thread:RxComputationThreadPool-2

Uppercase done on thread:main
Final value is:OBSERVEON:Thread:RxComputationThreadPool-3

Conclusion:

As we can see, in case of subscribeOn , irrespective of where it is called, both methods toUpper & printVal runs in a seperate thread. i.e. Threading applies to all operation (upstream as well as downtream) 

Whereas in case of observeOn, toUpper  is running in "main" thread & printVal  runs in a seperate thread, as we have called observeOn after the transform. i.e. Threading applied to downstream operations.

Now if we call the observeOn before the map:

Observable.just("observeOn Two").observeOn(Schedulers.computation()).map(ObsSubsEx::toUpper).subscribe(ObsSubsEx::printVal);

Then we can see the methods run in a seperate thread

Uppercase done on thread:RxComputationThreadPool-4
Final value is:OBSERVEON TWO:Thread:RxComputationThreadPool-4

Further Reading:


http://tomstechnicalblog.blogspot.com/2016/02/rxjava-understanding-observeon-and.html

https://www.aanandshekharroy.com/articles/2018-01/rxjava-flowables

https://proandroiddev.com/understanding-rxjava-subscribeon-and-observeon-744b0c6a41ea

https://dzone.com/articles/server-sent-events-with-rxjava-and-sseemitter

Code Link in Github:





Wednesday, August 19, 2020

Linux On Windows 10

 Windows 10 update version 2004 has come up with WSL 2 (Windows Subsystem for Linux ;Version 2). This features enable you to use Linux environment seamlessly from windows system.

Pre-requisite before install of WSL2.

To check Windows 10 version, follow below steps

  1. Open the command prompt or powershell window
  2. Type winver
  3. A pop-up will appear & show you windows update verion. Please check if the version is 2004 or not.
  4. Please update windows if the version is below 2004.

WSL 2 enablement also requires the Virtualization to be enabled. 

To check if Virtualization is enabled or not follow below steps:

  1. Open Task Manager--> Goto Performance Tab --> Click on CPU.
  2. Now check if Virtualization is enabled or not.

Now proceed to install WSL2 as instructed in below link:

https://www.youtube.com/watch?v=D7Em1wjMiak&t=179s

Monday, August 17, 2020

Medium Post Unlock

 Medium has lock on posts if you are not a member. You can read upto 3 medium posts per month freely.

To read the Medium posts without being a member ,the trick is copy the url & open in incognitio tab for Chrome browser.

Thursday, June 4, 2020

Useful Links



Useful Commands:
Run Spring Boot application in a port assigned dynamically

mvn spring-boot:run -Dspring-boot.run.arguments=--server.port=8080

Monday, June 1, 2020

Git Commands

Below is the set of commands needs to be executed to update code from local system to Github repository 

git config --global user.name "<Your Name>"

git config --global user.email <E-mail id>

create folder: e.g. Test

Navigate to folder

git init

git remote add origin <github url>

git pull origin master

Make changes to the folder (Test)

git status

git add .

git commit -m "Test Comment"

git push origin master

In Windows a good alternative is TortoiseGit. It can be downloaded from below link:

https://tortoisegit.org/

Friday, May 29, 2020

Changing the context name in Tomcat

Sometimes, it may happen, you want to provide the application context name  different from the WAR file name while deploying in Tomcat.

Scenario:

WAR Name: ABC.war
The default url becomes: http://localhost:8080/ABC

Want to access the url as: http://localhost:8080/XYZ

There are 2 options available to do this change

Option #1:
Here is the steps that can be followed:

1. Navigate to <TOMCAT_HOME>/conf/server.xml
2. Goto the Host section & do the below changes
3. Change it to below:

 <Host name="localhost"  appBase="webapps"
      unpackWARs="false" autoDeploy="false" deployOnStartup="false">    

Please note, unpackWARs , deployOnStartup, autoDeploy all three should be marked as false, else 2 folders will be generated one with name ABC & another XYZ.

4. Add the context changes under Host

 <Host name="localhost"  appBase="webapps"
      unpackWARs="false" autoDeploy="false" deployOnStartup="false">  
<Context path="/XYZ" docBase="ABC.war"/>
 <!-- other preexisting configuration-->
<Host>
5. Start Tomcat going to <TOMCAT_HOME>/bin/startup.bat (windows) or <TOMCAT_HOME>/bin/startup.sh in Linux environment.

Option#2:
  1. Explode the ABC.war (unzip the WAR file)
  2. Place the exploded WAR in a folder outside Tomcat Directory (e.g. D:\mywebapps)
  3. So, now the exploded WAR path will be D:\mywebapps\ABC
  4. Create an xml file in <TOMCAT_HOME>/conf/Catalina/localhost named XYZ.xml (the name of expected context)
  5. Now add the below line in XYZ.xml
  6. <Context path="/XYZ" docBase="D:/mywebapps/ABC"/>
  7. docBase refer to the path where exploded WAR is placed.
  8. Start Tomcat going to <TOMCAT_HOME>/bin/startup.bat (windows) or <TOMCAT_HOME>/bin/startup.sh in Linux environment.




Wednesday, March 4, 2020

Thread Dump in Java

Steps to take thread dump in Java in Windows

1. Download PSTools from below link
https://docs.microsoft.com/en-us/sysinternals/downloads/pstools
2. Use below command to take thread dump:

psexec -s <Path_to_JDK_bin_folder>\jstack.exe -l <process_id> ><PATH_WITH_FILE_NAME_FOR_DUMP>

e.g.
psexec -s D:\jdk1.8.0_171\bin\jstack.exe -l 319732 >D:\dump.txt

3. Download IBM Thread Dump Analyzer(TDA) from below link
https://public.dhe.ibm.com/software/websphere/appserv/support/tools/jca/jca465.jar

4. Double click on the jar to open
5. Click on File-->Open Thread Dumps.
6. Choose the thread dump file
7. Click on Analysis-->Thread Status Analysis
 


Details on TDA can be found here 



Simplifying Third-Party API Integration in Java with OpenFeign

Integrating third-party APIs is a common requirement in modern applications. Traditionally, developers rely on tools like Apache HttpClient ...