Batch Processing in Mule

Batch processing is useful when you want to process records in bulk

Here use case is to upload the student's records in bulk from CSV file to oracle database.



1. Create a mule project in studio and add Batch element to the canvas batch processing has three phases Input, Load and dispatch, Process, and OnComplete

  • Input: It is an optional part of batch process configuration. This is the initiator for the batch process to start, it can be file adaptor which can poll for file or poll component which will look for any records from a database or inbound connectors. During this phase, no splitting or aggregation is performed by mule. it just acts upon the message payload. you can perform any transforms before the message gets feed into process phase.
  • Load and Dispatch: This is the implicit phase of batch processing and you don't need to configure anything. It creates a batch job instance, This is a phase where mule splits the message into the record by collection splitter, Mule creates a persistent queue and associates it to the new batch job instance.
  • Process: This is the required phase of the batch job Mule begins asynchronous processing of the records in the batch within this required phase, each record moves through the message processors in the first batch step, then is sent back to queue before sending into next batch steps and so on until every record has passed through every batch step. Each record keeps track of the stage it has been processed through while it sits in the queue. Batch job instance does not wait for all its queued records to finish processing in one batch step 
  • On complete: This is the final phase of batch processing, you can configure the steps to send reports of how many records are processed successfully /failed.
2. In the input phase, add a file connector to read a student.csv file
student.csv

Id,Name,Course
1,Gopi,Java
2,Kar,Testing

3. Add TransformMessage processor to convert the csv format to collection list of students records.

4. Add the database connector in batch step to insert records into students table.



you can download the project from github

Comments

Post a Comment

Popular posts from this blog

RingCentral - How to use ringcentral Webhook features for events in Java

Image
Ring central cloud communication Platform provides webhook to notify you for any events like voicemail, SMS from your account.In this blog we will see how to set up you application to receive notifications from ringcentral platform through webhook. I am using jetty server to run my webserver which is listening to the port 5000. Server server = new Server( 5000 ) ; server.setHandler( new WebhookServer()) ; server.start() ; server.join() ; override the handle method to set status to 200 OK to return success response from webserver. public void handle(String target , Request baseRequest , HttpServletRequest request , HttpServletResponse response) throws IOException , ServletException { response.setStatus(HttpServletResponse.SC_OK) ; response.setHeader( "Validation-Token" , request.getHeader( "Validation-Token" )) ; if (request.getMethod() == "POST" ) { String body = request.getReader().lines().collect(java.util.stream....
Read more

Hello world project in Spring

This is My first post on Spring. Happy Learning !! 1.Download Eclipse from https://www.eclipse.org/downloads/ 2. Once you install the Eclipse navigate to Help->Eclipse Marketplace and download Spring tools Spring uses IOC (Inversion of Control) container to create, initialize, manages objects and you don't need to create the java objects from the class which avoids tight coupling. In this blog, you will learn about the different ways of creating the java objects through spring core container IOC. Create a java project Create a bean class Hello.java- Spring uses setter methods to initialize the objects package com.practice; public class Hello { public Hello() { System.out.println("Hello constructor"); } public String message; public void getMessage() { System.out.println("Your Message"+ message); } public void setMessage(String message) { this.message = message; } } Create a spring bean configuration...
Read more

Building and unit testing Mule project with Maven

Image
Mule is a project management tool to manage dependency and you can use it to develop a project and test it. 1. Create a mule project and select use Maven checkbox to enable maven support to the project and click finish. Maven will start downloading the dependency it requires to build a project based on pom.xml example:   project1 2. Create a mule flow by adding a http  connector and set payload 3. Add the http listener configurations details as below         Host: Localhost         Port :8081         path: /hello        Allowed Method: GET 4. Configure the Set Payload component to set the value to "Hello World!" Once you add http connector Maven adds the dependency in pom.xml and  automatically download its required dependencies         <dependency>           ...
Read more