Advanced Message Security with IBM WebSphere MQ

In these days I’m involving with some serious business with IBM WebSphere MQ …  🙂 . This is the last part of my IBM WebSphere MQ project. So I thought to share my experience with you.

IBM WebSphere MQ has a separate component called “Advanced Message Security aka AMS”, with this feature it enables you to encrypt message and puts it in your queue. Just don’t get me wrong , It doesn’t encrypt message when it leaves your application . Encryption happens when message reaches the IBM WebSphere MQ server. So you don’t need to break your existing code. Only thing you need to make sure , You have to install AMS component in your IBM MQ server.

Let’s say I’m in an intranet and my user name is “herath”(192.168.10.1) and I want to send a message to MQ server (192.168.10.2) . So in server there is an user “bobmquser” who has permission to read this message. So also my mq server in diffrent domain ( WMQSERVER\bobmqserver ) and I am in another domain (XXXX\herath). Therefore I’m going to connect as WMQSERVER\MUSR_MQADMIN. In this scenario I will set write permission to MUSR_MQADMIN and read permission to bobmquser.

Intranet

Intranet

First create an queue manager called QM_AMSTest , then local queue called XYZ and server-client channel called ABC and set it user as MUSR_MQADMIN

channel

channel

To test this message security create an “ALIAS” queue

Alias queue

Alias queue

Go to MQ server then select the queue manager (QM_AMSTest). First you need to set inquire and connect permissions to both users on queue manager.

Manager Authority

Manager Authority

Set Read and Write permissions

Set inquire and connect permissions

Go to your queue and right click and select manage authority records and give read and write permissions.

Manage authority records

set read and write permissions

Now we need to make these users trust each others ( XXXX\herath and WMQSERVER\bobmquser ). To do that we are going to user self-signed certificates.

So in this case we need to create key repositories. I am going to use C# console application to put message. Therefore I’m going to use “CMS” type key repositories.

Therefore i creat one key database for IBM MQ Server (192.168.10.2) WMQSERVER\bobmquser in following path (path doesn’t matter) D:\WebsphereKeyRepositary\AMSKeyStore\key.kdb and self-signed certificate for bobmquser

bobmquser's certificate

bobmquser’s certificate

repeat for the XXXX\herath ( 192.168.10.1). Key repository file path :- D:\AMSKeyRepo\key.kdb (path doesn’t matter, you can create this *.kdb file where ever you want )

Herath's certificate

Herath’s certificate

After creating these self-signed certificates . You need to generate *.arm file of each certificate. Then export certificates into each others key.kdb. You can use IBM Key management tool to exchange them. If you want to know how to exchange self-singed certificates please go through my “Exchanging self-signed certificates ” part in Transport level security with IBM WebSphere MQ in .net environment post.

To enable message security we need to create client keystore and server keystore in thier respective .mqs folders in thier %homepath%. You can use mkdir command in CMD to create .mqs folder.

Client's keystore

Client’s keystore

server's keystore

server’s keystore

Now we need to add a security policy which decides how to sign and encrypt message with self-signed certificates’ keys

Security Policy

Security Policy

So time to test our message security

using IBM.WMQ;
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace WebsphereMQ
{
    public class MessageProcess
    {

        const String connectionType = IBM.WMQ.MQC.TRANSPORT_MQSERIES_CLIENT;
        const String qManager = "QM_AMSTest";
        const String hostName = "192.168.10.2";
        const String channel = "ABC";
        const String port = "1501";

        MQQueue mqQueue; MQMessage mqMsg; MQGetMessageOptions mqGetMsgOpts;

        public Hashtable init()
        {
            Hashtable properties = new Hashtable();
            properties.Add(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES_CLIENT);
            properties.Add(MQC.HOST_NAME_PROPERTY, hostName);
            properties.Add(MQC.PORT_PROPERTY, port);
            properties.Add(MQC.CHANNEL_PROPERTY, channel);
            return properties;
        }

        public void RetrieveMessages()
        {
            try
            {
                Hashtable connectionProperties = init();
                MQQueueManager qMgr = new MQQueueManager(qManager, connectionProperties);
                mqQueue = qMgr.AccessQueue("XYZ", MQC.MQOO_INPUT_EXCLUSIVE | MQC.MQOO_FAIL_IF_QUIESCING);

                bool isContinue = true;
                int intRecordCounter = 1;

                while (isContinue)
                {

                    mqMsg = new MQMessage();
                    mqGetMsgOpts = new MQGetMessageOptions();

                    try
                    {
                        mqQueue.Get(mqMsg, mqGetMsgOpts);
                        var format = mqMsg.Format;
                        string message = mqMsg.ReadString(mqMsg.MessageLength);

                        if (mqMsg.Format.CompareTo(MQC.MQFMT_STRING) == 0)
                        {
                            Console.WriteLine("\n " + intRecordCounter + ") " + message);
                        }
                        else
                        {
                            Console.WriteLine("\n Non-text message");
                        }

                        // Counter Variable
                        intRecordCounter = intRecordCounter + 1;
                    }
                    catch (MQException mqe)
                    {
                        if (mqe.ReasonCode == 2033)
                        {
                            Console.WriteLine("No message available");
                        }
                        else
                        {
                            Console.WriteLine("MQException caught: {0} - {1}", mqe.ReasonCode, mqe.Message);
                        }

                        break;
                    }
                }

                Console.WriteLine("Read completed. Close the window");
                Console.ReadLine();

            }
            catch (MQException ex)
            {
                Console.WriteLine("WebSphere MQ error occurred: {0}", ex.ToString());
            }
            catch (System.Exception ex)
            {
                Console.WriteLine("System error occurred: {0}", ex.ToString());
            }

        }

        public void PutMessage()
        {
            try
            {
                Hashtable connectionProperties = init();
                MQQueueManager qMgr = new MQQueueManager(qManager, connectionProperties);
                MQQueue queue1 = qMgr.AccessQueue("XYZ", MQC.MQOO_OUTPUT | MQC.MQOO_FAIL_IF_QUIESCING);

                for (int i = 0; i < 5; i++)
                {
                    MQMessage msg = new MQMessage();
                    msg.Format = MQC.MQFMT_STRING;
                    msg.Encoding = 1208;
                    msg.Persistence = MQC.MQPER_PERSISTENT;
                    msg.CharacterSet = 1208;
                    msg.WriteString("Test Message " + i.ToString());
                    queue1.Put(msg);

                }

                Console.WriteLine("Write completed. Close the window");
            }
            catch (MQException ex)
            {
                Console.WriteLine("WebSphere MQ error occurred: {0}", ex.ToString());
            }
            catch (System.Exception ex)
            {
                Console.WriteLine("System error occurred: {0}", ex.ToString());
            }
        }
    }
}

Call the PutMessage() method from your Main method and run this application in your client machine. In my case I need to run this in XXXX\herath (192.168.10.1) pc. So it will put 5 messages at a time.

Messages in Alias queue

Messages in Alias queue

You can view these messages from alias queue and here it is with data encryption. So if bobmquser wants to retrieve these , Then go to the MQ server installation (see the CMD title bar ) and find the amqsget.exe  and execute like following image.

amqsget.exe

amqsget.exe

Remarks :-

  • When you insert message use WriteString() and if you use WriteUTF then when you retrieve messages it will come with some extra characters.
  • When you put messages in to queue, set the charset code ex:- 1208 for UTF-8

 

Posted in C#, IBM WebSphere MQ | 1 Comment

Transport level security with IBM WebSphere MQ in .net environment

Before we make this transport level security we have to make an queue manager with proper configuration. So first of all make sure to have following things in your queue manager.

  • IBM web sphere installation server’s user should be in mqm user group
  • Create an queue called XYZ

create a channel called ADMIN.TLS.SVRCONN with TLS_RSA_WITH_AES_128_CBC_SHA256

Create a SSL Required Channel

Create a SSL Required Channel

and here I set user as MUSER_MQADMIN , This user must be within “mqm” user group

mqm user group's user

mqm user group’s user

Set the web sphere mq server’s key data store location

key.kdb file location

WMQ server’s key.kdb file location

Create a CMS type key store in Web Sphere MQ server IBM websphere mq insalltion folder/QMgrs/QM_SSLConnect/SSL  (192.168.10.2)

IBM WMQ server's key store

IBM WMQ server’s key store

key store password

key store password

Create a self signed certificate for server

label name must be in lower letter case and start with ibmwebspheremq

IBM WMQ server's selfsigned certificate

IBM WMQ server’s selfsigned certificate

IBM WebSphere MQ SSL Connection (4)

extract this entry to certificate

Extract certificate

Extract certificate

Repeat this for Client side and create key store in SSL folder

Key store in client side

Key store in client side

here My user name is “herath” so label name must be ibmwebspherewebmqherath

Client certificate

Client certificate

if you want to get current user, you can use “whoami” command in cmd in your client machine

get current user from command prompt

get current user from command prompt

Then extract the certificate as clientcert.arm

Exchanging self-signed certificates 

Exchange self-signed certificates

Exchange self-signed certificates

now we need to import these certificates in to their respective key stores

Import client side self-signed ( clientcert.cert  )  certificate to server’s key-store

import client side certificate to server key store

import client side certificate to server key store

when it asks to enter label give ibmwebspheremqherath ( in your case your user name ibmwebspheremqxxx, xxx is your user name)

import client self-signed certificate

import client self-signed certificate

 

Repeat the same thing for server’s self-signed certificate in client side

import server's self-signed certificate

import server’s self-signed certificate

server's self-signed certificate label

server’s self-signed certificate label

After you complete these things , Then click the “Refresh SSL”

Refresh SSL

Refresh SSL

here is the code for connecting to SSL enabled web sphere channel


using IBM.WMQ;
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace WebsphereMQ
{
            public class SSLConnectionTest
        {
            const String connectionType = IBM.WMQ.MQC.TRANSPORT_MQSERIES_CLIENT;
            const String qManager = "QM_SSLConnect";
            const String hostName = "192.168.10.2";
            const String channel = "ADMIN.TLS.SVRCONN";
            const String port = "1480";
            const String sslKeyRepository = @"C:\Program Files (x86)\IBM\WebSphere MQ\ssl\key";
            const String cipherSpec = "TLS_RSA_WITH_AES_128_CBC_SHA256";
            const String cipherSuite = "SSL_RSA_WITH_AES_128_CBC_SHA256";

            public Hashtable init()
            {
                Hashtable properties = new Hashtable();

                properties.Add(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES_CLIENT);
                properties.Add(MQC.HOST_NAME_PROPERTY, hostName);
                properties.Add(MQC.PORT_PROPERTY, port);
                properties.Add(MQC.CHANNEL_PROPERTY, channel);

                properties.Add(MQC.SSL_CERT_STORE_PROPERTY, sslKeyRepository);
                properties.Add(MQC.SSL_CIPHER_SUITE_PROPERTY, cipherSuite);
                properties.Add(MQC.SSL_CIPHER_SPEC_PROPERTY, cipherSpec);

                return properties;
            }

            public void TestSSLConnection()
            {
                try
                {
                    Hashtable connectionProperties = init();
                    MQQueueManager qMgr = new MQQueueManager(qManager, connectionProperties);
                    MQQueue queue1 = qMgr.AccessQueue("XYZ", MQC.MQOO_OUTPUT | MQC.MQOO_FAIL_IF_QUIESCING);
                    MQMessage msg = new MQMessage();
                    msg.WriteUTF("Test Message");
                    queue1.Put(msg);

                }
                catch (MQException ex)
                {
                    Console.WriteLine("WebSphere MQ error occurred: {0}", ex.ToString());
                }
                catch (System.Exception ex)
                {
                    Console.WriteLine("System error occurred: {0}", ex.ToString());
                }
            }

            public void TestSSLConnectionWithCCDT()
            {
                try
                {
                    Environment.SetEnvironmentVariable("MQCHLLIB", @"D:\MQCCDT\QM_SSLConnect\");
                    Environment.SetEnvironmentVariable("MQCHLTAB", "AMQCLCHL.TAB");

                    Hashtable props = new Hashtable();
                    props.Add(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES_CLIENT);
                    props.Add(MQC.SSL_CERT_STORE_PROPERTY, sslKeyRepository);
                    MQQueueManager qMgr = new MQQueueManager("QM_SSLConnect", props);

                    MQQueue queue1 = qMgr.AccessQueue("XYZ", MQC.MQOO_OUTPUT | MQC.MQOO_FAIL_IF_QUIESCING);
                    MQMessage msg = new MQMessage();
                    msg.WriteUTF("Test Message");
                    queue1.Put(msg);
                }
                catch (MQException ex)
                {
                    Console.WriteLine("A WebSphere MQ error occurred: {0}", ex.ToString());
                }
                catch (System.Exception ex)
                {
                    Console.WriteLine("A System error occurred: {0}", ex.ToString());
                }
            }

        }
}

I included TestSSLConnectionWithCCDT() method to connect with CCDT file. If you want to connect with CCDT , first you need to set client channel and then get the CCDT file into client machine. (please see my previous post to connect with CCDT )

put message in to client through .net application with SSL enabled channel

put message in to client through .net application with SSL enabled channel

Posted in IBM WebSphere MQ | Tagged , , , | 2 Comments

Invoke C# console application on new message arrival in IBM WebSphere MQ

Before going to bed I thought to post this post. After couple days of struggling with IBM web sphere mq and other solutions on internet I could able to create a workable solution today… 🙂

In this post I’m going to tell you how to trigger simple c# console application when new message arrivals on MQ (IBM websphere MQ).

Lets say we have an queue manager called “TriggerTest” on 192.168.10.2 and it’s TCP listener port is 1436 and server – connection channel called “SVRCOMMUNICATION”.

 server - connection channel

server – connection channel

In this queue manager we have an queue called “LocalQueue” and we want to invoke an application when new message comes to this queue.

So first of all to monitor this local queue we need to create an initiation queue. This must be type of SYSTEM.DEFAULT.INITIATION.QUEUE.

Initiation queue

Initiation queue

once you create the initiation queue , it looks like this

TriggerTest queue manager

TriggerTest queue manager

Then right click on the local queue and set these trigger properties. Process name is process definition “Process Name”. See the “Process Definition” image.

Trigger Properties

Trigger Properties

if you want to get more details on trigger types do visit this link http://www-01.ibm.com/support/docview.wss?uid=swg27015657&aid=1   . For the time being I’ll add these details.

  • FIRST: A trigger event occurs when the current depth of the triggered queue changes from 0 to 1.
  • EVERY: A trigger event occurs every time a message arrives on the triggered queue.
  • DEPTH: A trigger event occurs when the number of messages on the triggered queue reaches the value of the TRIGDPTH attribute.

[taken from the above link]

Now right click on the Process Definitions and add new process. This is going to  invoke our console application.

Process Definition

Process Definition

C# console application will read the local queue message and then writes that data in to a text file.

So far we have created initiation queue , process definition. But we need to get a service for monitoring this message communication. So we have to have monitor service.

There are two ways to start this monitor service. You can start monitor local queue from websphere server by using runmqtrm.exe . If you use your client machine then you need to use runmqtrc.exe. You can find these exe files on << IBM WebSphere Installation Directory>>/bin/

runmqtmc

runmqtmc

In this post I’m going to start runmqtmc.exe from command prompt (CMD).

SET MQSERVER=SVRCOMMUNICATION/TCP/192.168.10.2(1436)

runmqtmc -m TriggerTest -q InitQueue

set MQServer

set MQServer

set connection with channel and mq

set connection with channel and mq

once it successful

trigger monitor service

trigger monitor service

you can find more details from this link http://www-01.ibm.com/support/knowledgecenter/SSFKSJ_7.5.0/com.ibm.mq.dev.doc/q026970_.htm 

Put a message on queue with your mq explorer client ,

MQ Explorer client

MQ Explorer client

mq trigger monitor

mq trigger monitor

When new message arrives  this service will invoke c# console application and writes the message in to a text file.

IBMWEBSPHEREMQTRIGGER (4)

Here is my C# console application code which will read mq and puts message in to queue


using IBM.WMQ;
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
 
namespace ConnectWithCCDT
{
    class Program
    {
        static void Main(string[] args)
        {
            new Program().ConnectWithFiles();
          
        }
 
        public void ConnectWithFiles()
        {
            MQQueueManager qm = null;
            Environment.SetEnvironmentVariable("MQCHLLIB", @"D:\MQCCDT\");
            Environment.SetEnvironmentVariable("MQCHLTAB", "AMQCLCHL.TAB");
 
            try
            {
                Hashtable props = new Hashtable();
                props.Add(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES_CLIENT);
                qm = new MQQueueManager("TriggerTest", props);
                MQQueue queue1 = qm.AccessQueue("LocalQueue", MQC.MQOO_INPUT_AS_Q_DEF + MQC.MQOO_FAIL_IF_QUIESCING);
 
                MQMessage message = new MQMessage();
                queue1.Get(message);
                string queueMsg = message.ReadString(message.MessageLength);
 
                StreamWriter sw = new StreamWriter(@"D:\aa.txt", true);
                string data = queueMsg + "    " + DateTime.Now.ToString();
                sw.WriteLine(data);
                sw.Flush();
                sw.Dispose();
 
                Console.WriteLine("Message from Queueu {0}", data);
 
                queue1.Close();
                qm.Disconnect();
                Console.WriteLine("Success.");
            }
            catch (Exception ex)
            {
                Console.Write(ex);
            }
 
        }
       
    }
}

I connected MQ with AMQCLCHL.TAB file. If you need more information please read my previous blog post https://2freeclear.wordpress.com/2014/11/05/ibm-websphere-mq-connection-with-ccdt-file/

 

Posted in C#, IBM WebSphere MQ | Tagged , , , , | 3 Comments

IBM WebSphere MQ Connection with CCDT file

With this post I’m going to tell you about how to configure and implement connection between IBM MQ version 7.5  with ccdt file (client channel definition table)… So I’m going to explain you about Server-Client MQ model.

First of all we need to have a privileged user in our MQ server (here after I refer IBM websphere MQ as simply MQ).

ibm-websphere-mq-1

MQ user

 

just make sure you grant that user to MQ administrator permissions. If your current user hasn’t enough permission then you can log in with user with MQ administrator permissions.

Once you have done with your user permissions , open MQ in administrator in administrator mode. Let’s create a message queue manager in MQ. We can call it QM_CCDTTest , here you can what ever you like.

Create Message Queue

Create Message Queue

next important step is setting up listener’s port. You will connect to this queue through this port number. You can scan current ports with “netstat -an” in your command prompt (Windows CMD). By the time being we set this as 1456

Listener's port

Listener’s port

ok next thing is creating a local queue

Create Local queue

Create Local queue

pretty easy isn’t it ??? just make sure to select Local Queue

Time to create a channel. This one should be server-connection channel ( NOT Server channel )

create a server connection channel

create a server connection channel

create a Channel

create a Channel

There are few things to remember when you  create server connection channel you need to set following properties in channel properties and these things

  • Give uppercase name and you need to set this same name in client connection
  • Set MCA User Id
  • set SSL authentication
set MCA user id

set MCA user id

for this option you can give previously created MQ admin privileged user name . So in my case I set it this name.

SSL property

SSL property

if you are not going to use ssl authentication just set it as “Optional”.

After this , create client connection. This will be going inside the CCDT file and you have to use the same name you already set for server channel ( need to be in upper case letter and same as server channel )

Create client connection

Create client connection

create client channel

create client channel

set these properties on the created client connection .

client connection properties

client connection properties

In the connection name you need to give host name which your MQ installed and Listener’s port number as shown in the image.

Ok then come to the next level – Channel authentication. By default there will be record for blocking *MQADMIN.

Channel authentication

Channel authentication

So we are almost done with configuration. Time to fetch the CCDT file. Go to your MQ installation directory. Most probably it will be something like this

CCDT file path

CCDT file path

copy AMQCLCHL.TAB file to your client computer. So we are going to refer this file and file path to make connection between MQ server. I put this file in my D:\MQCCDTTEST folder you can keep it where ever you like . Only thing you need to know is set that path correctly in your environment variable.

My CCDT file path

My CCDT file path

I’m going to show you two ways connecting with MQ server one with C# application and other one with MQ explorer

so let’s try with MQ explorer

open your MQ explorer in client computer

MQ explorer connection with CCDT

MQ explorer connection with CCDT

Right click on Queue manager and create a new Queue . Give the same name you already set in your MQ . In my case QM_CCDTTest. Then select connect using with CCDT file (second option) and choose the AMQCLCHL.TAB file. Once you complete this one you will be ended up with following screen

MQ Explorer client connection with CCDT

MQ Explorer client connection with CCDT

From here on, you can browse your queue’s message , put message …etc

Connection with C# console application

First you need to add amqmdnet.dll  reference to your project. You can find this dll in MQ explorer installation directory

amqmdnet.dll 

amqmdnet.dll

Create a console application and following code to your project. The only thing you need to connect to MQ is setting environment variables

Environment.SetEnvironmentVariable(“MQCHLLIB”, @”D:\MQCCDT\”);
Environment.SetEnvironmentVariable(“MQCHLTAB”, “AMQCLCHL.TAB”);

 


using IBM.WMQ;
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ConnectWithCCDT
{
    class Program
    {
        static void Main(string[] args)
        {
            new Program().ConnectWithFiles();
            Console.ReadLine();
        }

        public void ConnectWithFiles()
        {
            MQQueueManager qm = null;
            Environment.SetEnvironmentVariable("MQCHLLIB", @"D:\MQCCDT\");
            Environment.SetEnvironmentVariable("MQCHLTAB", "AMQCLCHL.TAB");

            try
            {
                Hashtable props = new Hashtable();
                props.Add(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES_CLIENT);
                qm = new MQQueueManager("QM_CCDTTest", props);
                MQQueue queue1 = qm.AccessQueue("LocalQueue", MQC.MQOO_OUTPUT | MQC.MQOO_FAIL_IF_QUIESCING);

                MQMessage msg = new MQMessage();
                msg.WriteUTF("Test Message");
                queue1.Put(msg);
                queue1.Close();
                qm.Disconnect();
                Console.WriteLine("Success.");
            }
            catch (Exception ex)
            {
                Console.Write(ex);
            }

        }
    }
}

once you execute this code you can see it on the queue

MQ Explorer final message

MQ Explorer final message

Posted in IBM WebSphere MQ | Tagged , | 2 Comments

MVC WebGrid pagination and sorting with Entity Framework

So after some long time I had to get in to some old problem. But this time I figured it out. So lets go through this “MVC WebGrid pagination and sorting with Entity Framework”

With this implementation, I’m able to do the pagination in database level. That means this doesn’t load all the records at once , instead of loading whole result set it will load batch of records. You can check this with Sql server profiler.

For this solution I have used following technologies

I use visual studio as IDE and download these frameworks / libs by using nuget packages. For this example I used Northwind database  .

Download the project

Data Layer

Add Northwind.edmx file to data layer

Northwind.edmx

Northwind.edmx

and here I add another view-model called “FilterModel” which I’m going to use it in my Pager controller’s Index.cshtml

 public class FilterModel
 {
    public int Page { get; set; }
    public int PageSize { get; set; }
    public int TotalCount { get; set; }
    public string SearchText { get; set; }
    public string Sort { get; set; }
    public string Sortdir{ get; set; }
    public List<Order> DataModel { get; set; }
 }

Ok… That’s it Now we have done with our Data. So time to move on to service layer…

Service Layer [ Business Layer ]

Order Service

Service Layer


 public class OrderService
    {
        public FilterModel GetFilterModel(FilterModel model)
        {
            model.Page = model.Page == 0 ? 1 : model.Page;
            model.PageSize = model.PageSize == 0 ? 5 : model.PageSize;
            model.SearchText = string.IsNullOrEmpty(model.SearchText) ? model.SearchText : model.SearchText.Trim();
            model.Sort = string.IsNullOrEmpty(model.Sort) ? "CustomerID" : model.Sort;
            model.Sortdir = string.IsNullOrEmpty(model.Sortdir) ? "ASC" : model.Sortdir;
            int skipValue = (model.Page - 1) * model.PageSize;

            Func<Order, bool> prediction = x => string.IsNullOrEmpty(model.SearchText) ||
                                            model.SearchText == x.CustomerID ||
                                            model.SearchText == x.ShipAddress ||
                                            model.SearchText == x.ShipName ||
                                            model.SearchText == x.ShipCity;

            if (model.Sortdir.ToLower() == "asc")
            {
                model.DataModel = new NorthwindEntities().Orders
                                .Where(prediction)
                                .OrderBy(GenericEvaluateOrderBy<Order>(model.Sort)).Skip(skipValue).Take(model.PageSize).ToList();
            }
            else
            {
                model.DataModel = new NorthwindEntities().Orders
                                .Where(prediction)
                                .OrderByDescending(GenericEvaluateOrderBy<Order>(model.Sort)).Skip(skipValue).Take(model.PageSize).ToList();
            }

            model.TotalCount = new NorthwindEntities().Orders
                                .Where(prediction)
                                .Count();

            return model;
        }

        public Func<TSource, object> GenericEvaluateOrderBy<TSource>(string propertyName)
        {
            //this will return p=>p.<< your proeprty name >>
            var type = typeof(TSource);
            var parameter = Expression.Parameter(type, "p");
            var propertyReference = Expression.Property(parameter, propertyName);
            return Expression.Lambda<Func<TSource, object>>(propertyReference, new[] { parameter }).Compile();
        }
    }

GetFilterModel – I call this method from my action method in controller and pass FilterModel. You can see in first couple of lines I’m checking its properties and manually assigning values. just because of when my Index.cshtml loads first time I have to have some default values.

To implement pagination and sorting we have to use some kind of Linq query like this

var result = context.<< table name >>.where( << your search text >> ).OrderBy or OrderDescendingBy ( << sort column name >> ).Skip( << (Current page number-1) * rows per page >> ).Take( << rows per page >> ).ToList<< table name >>()

by looking at this code you can see when user tries to search something or when he changes the sort column we need to change this query.

So I used this code block to get Where condition

Func<Order, bool> prediction = x => string.IsNullOrEmpty(model.SearchText) ||
                                model.SearchText == x.CustomerID ||
                                model.SearchText == x.ShipAddress ||
                                model.SearchText == x.ShipName ||
                                model.SearchText == x.ShipCity;

ok lets move to order by clause. I had following problems

  • How can I sort when my sort column gets change
  • how can I handle Ascending and Descending

GenericEvaluateOrderBy<TSource>(string propertyName) – Suppose what will happen If I don’t use this method… Then I’m running to pretty much if else || switch case code block. So instead of implementing column name with conditions , I passed column name as a variable in to my Linq query then I’m able to reduce my code block.

( actually I got some help from this web article so much of credit goes to the author of article https://www.simple-talk.com/dotnet/.net-framework/dynamic-linq-queries-with-expression-trees/ )

 Controller

Add a controller to your mvc project. I named this as a Pager you can give what ever your prefer

PageController

PageController

public class PagerController : Controller
{
        public ActionResult Index(FilterModel model)
        {
            OrderService service = new OrderService();
            return View(service.GetFilterModel(model));
        }
}

Views

Next thing is a adding a view to the project.

Index.cshtml

Index.cshtml

Here I need you to tell you some couple of things, as I told you earlier I used bootstrap ui , font-awsome and jquery .Actually you can get these stuffs through Nuget package manager in visual studio or else download from their respective sites.

once you get these things make sure you bundle them in BundleConfig.cs you can find this class in App_start folder

BundleConfig.cs

BundleConfig.cs

and here is the content of class for further clarification…


public class BundleConfig
    {
        // For more information on bundling, visit http://go.microsoft.com/fwlink/?LinkId=301862
        public static void RegisterBundles(BundleCollection bundles)
        {
            bundles.Add(new ScriptBundle("~/bundles/jquery").Include(
                        "~/Scripts/jquery-{version}.js"));

            // Use the development version of Modernizr to develop with and learn from. Then, when you're
            // ready for production, use the build tool at http://modernizr.com to pick only the tests you need.
            bundles.Add(new ScriptBundle("~/bundles/modernizr").Include(
                        "~/Scripts/modernizr-*"));

            bundles.Add(new ScriptBundle("~/bundles/bootstrap").Include(
                      "~/Scripts/bootstrap.js",
                      "~/Scripts/respond.js"));

            bundles.Add(new StyleBundle("~/Content/css").Include(
                      "~/Content/bootstrap.css",
                      "~/Content/site.css"));

            bundles.Add(new StyleBundle("~/Content/font-awesome").Include("~/Content/font-awesome.css"));
        }
    }

Content folder

In my _Layout.cshtml


<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>@ViewBag.Title - My ASP.NET Application</title>
    @Styles.Render("~/Content/css")
    @Styles.Render("~/Content/font-awesome")
    @Scripts.Render("~/bundles/modernizr")
    @RenderSection("styles", required: false)
</head>
<body>
    <div class="navbar navbar-inverse navbar-fixed-top">
        <div class="container">
            <div class="navbar-header">
                <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
                    <span class="icon-bar"></span>
                    <span class="icon-bar"></span>
                    <span class="icon-bar"></span>
                </button>
                @Html.ActionLink("Application name", "Index", "Home", null, new { @class = "navbar-brand" })
            </div>
            <div class="navbar-collapse collapse">
                <ul class="nav navbar-nav">
                    <li>@Html.ActionLink("Home", "Index", "Home")</li>
                    <li>@Html.ActionLink("About", "About", "Home")</li>
                    <li>@Html.ActionLink("Contact", "Contact", "Home")</li>
                </ul>
                <ul class="nav navbar-nav navbar-right">
                    <li class="navbar-text">Hello, @User.Identity.Name!</li>
                </ul>
            </div>
        </div>
    </div>
    <div class="container body-content">
        @RenderBody()
        <hr />
        <footer>
            <p>&copy; @DateTime.Now.Year - My ASP.NET Application</p>
        </footer>
    </div>

    @Scripts.Render("~/bundles/jquery")
    @Scripts.Render("~/bundles/bootstrap")
    @RenderSection("scripts", required: false)
</body>
</html>

Views –> Pager –> Index.cshtml


@using PaginationTest.Models
@model FilterModel

@{
    ViewBag.Title = "Index";
    Layout = "~/Views/Shared/_Layout.cshtml";
}

@section scripts{

    <script type="text/javascript">
        var gridSort = '@Model.Sort';
        var gridSortDir = '@Model.Sortdir';
    </script>

    <script src="~/Scripts/AppScripts/PagerScript.js"></script>
}

@section styles
{
    <style type="text/css">
        .space-box {
            margin: 10px;
        }
    </style>

}

@{
    WebGrid grid = new WebGrid(null, rowsPerPage: Model.PageSize, canPage: true, defaultSort: "CustomerID");
    grid.Bind(Model.DataModel, rowCount: Model.TotalCount, autoSortAndPage: false);
}

@using (@Html.BeginForm("Index", "Pager", FormMethod.Post, new { @id = "frmDetails" }))
{

    <div class="container">

        <div class="panel">
            <div class="panel-heading">
                <h3 class="panel-title">Order Details</h3>
            </div>
            <div class="panel-body">

                <div class="row">
                    <div class="col-md-4">

                        <div class="form-inline">
                            @Html.DropDownListFor(m => m.PageSize,
                                                    new List<SelectListItem>
                                                    {
                                                        new SelectListItem() { Text = "5", Value = "5" },
                                                        new SelectListItem() { Text = "10", Value = "10" },
                                                        new SelectListItem() { Text = "20", Value = "20" }
                                                    },
                                                    new { @class = "form-control" }

                                                )
                            <span>Rows per page</span>
                        </div>
                    </div>

                    <div class="col-md-2"><p>Records :@{ var x = (Model.Page * Model.PageSize) >= Model.TotalCount ? Model.TotalCount : (Model.Page * Model.PageSize);  } @x out of @Model.TotalCount</p></div>
                    <div class="col-md-2"></div>
                    <div class="col-md-4">
                        <div class="input-group">
                            <div class="input-group-addon"><span class="glyphicon glyphicon-search"></span></div>
                            <input class="form-control" type="text" name="SearchText" id="SearchText" value="@Model.SearchText" />
                        </div>
                    </div>
                </div>

                <div class="row space-box"></div>

                <div class="row">
                    @if (Model.DataModel != null && Model.DataModel.Count > 0)
                    {
                        @grid.GetHtml(tableStyle: "table table-bordered", columns:
                            grid.Columns(
                                            grid.Column("CustomerID", "Customer ID"),
                                            grid.Column("ShipName", "Ship Name"),
                                            grid.Column("ShipAddress", "Ship Address"),
                                            grid.Column("ShipCity", "Ship City")
                                        )
                            )
                    }
                    else
                    {
                        <div class="alert alert-info" role="alert">No Data</div>
                    }

                </div>

            </div>
            <div class="panel-footer">
                <div class="row">
                    <div id="gridPager" class="space-box">
                        @if (Model.DataModel != null && Model.DataModel.Count > 0)
                        {
                            @grid.PagerList(WebGridPagerModes.All)
                        }
                    </div>
                </div>

            </div>

        </div>

    </div>

}

You need to remember following steps are really important when you need to implement pagination and sorting

@{
    WebGrid grid = new WebGrid(null, rowsPerPage: Model.PageSize, canPage: true, defaultSort: "CustomerID");
    grid.Bind(Model.DataModel, rowCount: Model.TotalCount, autoSortAndPage: false);
}
  •  set the default sort column ( CustomerID )
  • set the total row count of your search query ( Model.TotalCount )

I hide default pager by using J query and implemented bootstrap pager But you need to have web gird pagination extension method to work this properly. This is the extension class and it is not mine I got it from this article  

Webgrid Extension

Webgrid Extension


public static class WebGridExtensions
{
    public static HelperResult PagerList(
        this WebGrid webGrid,
        WebGridPagerModes mode = WebGridPagerModes.NextPrevious | WebGridPagerModes.Numeric,
        string firstText = null,
        string previousText = null,
        string nextText = null,
        string lastText = null,
        int numericLinksCount = 5)
    {
        return PagerList(webGrid, mode, firstText, previousText, nextText, lastText, numericLinksCount, explicitlyCalled: true);
    }

    private static HelperResult PagerList(
        WebGrid webGrid,
        WebGridPagerModes mode,
        string firstText,
        string previousText,
        string nextText,
        string lastText,
        int numericLinksCount,
        bool explicitlyCalled)
    {

        int currentPage = webGrid.PageIndex;
        int totalPages = webGrid.PageCount;

        //int lastPage = totalPages - 1;
        int lastPage = (totalPages - 1) < 0 ? 0 : (totalPages - 1);

        var ul = new TagBuilder("ul");
        var li = new List<TagBuilder>();

        if (ModeEnabled(mode, WebGridPagerModes.FirstLast)) {
            if (String.IsNullOrEmpty(firstText)) {
                firstText = "First";
            }

            var part = new TagBuilder("li") {
                InnerHtml = GridLink(webGrid, webGrid.GetPageUrl(0), firstText)
            };

            if (currentPage == 0) {
                part.MergeAttribute("class", "disabled");
            }

            li.Add(part);

        }

        if (ModeEnabled(mode, WebGridPagerModes.NextPrevious)) {
            if (String.IsNullOrEmpty(previousText)) {
                previousText = "Prev";
            }

            int page = currentPage == 0 ? 0: currentPage - 1;

            var part = new TagBuilder("li") {
                InnerHtml = GridLink(webGrid, webGrid.GetPageUrl(page), previousText)
            };

            if (currentPage == 0) {
                part.MergeAttribute("class", "disabled");
            }

            li.Add(part);

        }

       if (ModeEnabled(mode, WebGridPagerModes.Numeric) && (totalPages > 1)) {
            int last = currentPage + (numericLinksCount / 2);
            int first = last - numericLinksCount + 1;
            if (last > lastPage) {
                first -= last - lastPage;
                last = lastPage;
            }
            if (first < 0) {
                last = Math.Min(last + (0 - first), lastPage);
                first = 0;
            }
            for (int i = first; i <= last; i++) {

                var pageText = (i + 1).ToString(CultureInfo.InvariantCulture);
                var part = new TagBuilder("li") {
                    InnerHtml = GridLink(webGrid, webGrid.GetPageUrl(i), pageText)
                };

                if (i == currentPage) {
                    part.MergeAttribute("class", "active");
                }

                li.Add(part);

            }
        }

        if (ModeEnabled(mode, WebGridPagerModes.NextPrevious)) {
            if (String.IsNullOrEmpty(nextText)) {
                nextText = "Next";
            }

            int page = currentPage == lastPage ? lastPage: currentPage + 1;

            var part = new TagBuilder("li") {
                InnerHtml = GridLink(webGrid, webGrid.GetPageUrl(page), nextText)
            };

            if (currentPage == lastPage) {
                part.MergeAttribute("class", "disabled");
            }

            li.Add(part);

        }

        if (ModeEnabled(mode, WebGridPagerModes.FirstLast)) {
            if (String.IsNullOrEmpty(lastText)) {
                lastText = "Last";
            }

            var part = new TagBuilder("li") {
                InnerHtml = GridLink(webGrid, webGrid.GetPageUrl(lastPage), lastText)
            };

            if (currentPage == lastPage) {
                part.MergeAttribute("class", "disabled");
            }

            li.Add(part);

        }

        ul.InnerHtml = string.Join("", li);

        var html = "";
        if (explicitlyCalled && webGrid.IsAjaxEnabled) {
            var span = new TagBuilder("span");
            span.MergeAttribute("data-swhgajax", "true");
            span.MergeAttribute("data-swhgcontainer", webGrid.AjaxUpdateContainerId);
            span.MergeAttribute("data-swhgcallback", webGrid.AjaxUpdateCallback);

            span.InnerHtml = ul.ToString();
            html = span.ToString();

        } else {
            html = ul.ToString();
        }

        return new HelperResult(writer => {
            writer.Write(html);
        });
    }

    private static String GridLink(WebGrid webGrid, string url, string text)
    {
        TagBuilder builder = new TagBuilder("a");
        builder.SetInnerText(text);
        builder.MergeAttribute("href", url);
        if (webGrid.IsAjaxEnabled) {
            builder.MergeAttribute("data-swhglnk", "true");
        }
        return builder.ToString(TagRenderMode.Normal);
    }

    private static bool ModeEnabled(WebGridPagerModes mode, WebGridPagerModes modeCheck)
    {
        return (mode & modeCheck) == modeCheck;
    }

}

Scripts

So we have made so far. This is the last section. Create a script file name PagerScript.js and put that in this folder path.

Pager Script

Pager Script


function getParameterByName(url, name) {
    name = name.replace(/[\[]/, "\\[").replace(/[\]]/, "\\]");
    var regex = new RegExp("[\\?&]" + name + "=([^&#]*)"),
        results = regex.exec(url);
    return results == null ? "" : decodeURIComponent(results[1].replace(/\+/g, " "));
}

function setArrowImages() {

    $('table th').each(function () {

        //get direction value
        var direction = getParameterByName($(this).find('a').attr('href'), 'sortdir');
        var header = $(this).find('a[href*="sortdir"]');
        header.html(header.html() + ' <i class="fa fa-sort pull-right" />');

        if (gridSort == getParameterByName($(this).find('a').attr('href'), 'sort')) {
            switch (gridSort) {
                case 'CustomerID': if (gridSortDir.toLowerCase() == "asc") { $(this).find('i').removeAttr('class').addClass('fa fa-sort-asc pull-right') } else { $(this).find('i').removeAttr('class').addClass('fa fa-sort-desc pull-right') }; break;
                case 'ShipName': if (gridSortDir.toLowerCase() == "asc") { $(this).find('i').removeAttr('class').addClass('fa fa-sort-asc pull-right') } else { $(this).find('i').removeAttr('class').addClass('fa fa-sort-desc pull-right') }; break;
                case 'ShipAddress': if (gridSortDir.toLowerCase() == "asc") { $(this).find('i').removeAttr('class').addClass('fa fa-sort-asc pull-right') } else { $(this).find('i').removeAttr('class').addClass('fa fa-sort-desc pull-right') }; break;
                case 'ShipCity': if (gridSortDir.toLowerCase() == "asc") { $(this).find('i').removeAttr('class').addClass('fa fa-sort-asc pull-right') } else { $(this).find('i').removeAttr('class').addClass('fa fa-sort-desc pull-right') }; break;
                default:
            }
        }

    });
};

$(function () {

    $('#gridPager ul').addClass('pagination');
    $('tfoot').hide();

    $('#PageSize').change(function () {

        $('#frmDetails').submit()
        return false;

    });

    $('#SearchText').change(function () {

        $('#frmDetails').submit()
        return false;

    });

    $('#gridPager a').click(function (e) {

        var form = $('#frmDetails');
        form.attr("action", this.href);
        $(this).attr("href", "javascript:");
        form.submit();

    });

    $('th a').click(function () {

        var form = $('#frmDetails');
        form.attr("action", this.href);
        $(this).attr("href", "javascript:");
        form.submit();

    });

    setArrowImages();

});

you may be wondering how I stopped default “GET” request… let me explain

$('th a').click(function () {

        var form = $('#frmDetails');
        form.attr("action", this.href);
        $(this).attr("href", "javascript:");
        form.submit();

    });

from this part I’m handling pagination. When user clicks pager we need to do a form submit (not a AJAX form submit It must be a fully form postback to server ). So I changed current form action. To do that I changed anchor tag href value. It is some thing like http://localhost:56896/?page=1 but at the same time I need to stop that html link click,  you can modify href value like this $(this).attr(“href”,”javascript:”) Thereby it doesn’t do anything when user clicks this link and it will not send this “GET” request but it will send “POST” request to the server in our case it will invoke our “Index” action method. If you feel some strange about this $(this).attr(“href”,”javascript:”) code then you can change it to $(this).attr(“href”,”#”). Because both of them do the same thing.

So I hope this post will help you too and this is the final result

If you want to know anything and share your knowledge with me please do comment. Thank you all have a nice day with nice web grid.

WebGrid pagination with bootstrap UI and EF

WebGrid pagination with bootstrap UI and EF

Download the project

Posted in MVC | Jquery | Webgrid | Pagination | Sorting | Entity Framework | Tagged , , | 4 Comments

Create nested html tables from complex JSON

Recently I came a cross to implement a nested tables from complex object ( object that has more children objects). So end users can toggle nested table and look inside in those child elements. To achieve this I serialized the object in to JSON string and a recursively created tables from its data. Here is my solution

Continue reading

Posted in Javascript | JSON | Jquery | Tagged , , , | 3 Comments