Saturday, August 12, 2017

Debian 9 - RabbitMQ Installation Notes

Add respository below into /etc/apt/sources.list
    deb http://www.rabbitmq.com/debian/ testing main
    apt-get update
Install
    apt-get install rabbitmq-server -y

Check Status
    service rabbitmq-server status

version check
    rabbitmqctl status | grep RabbitMQ 

Enable plugins required for rabbitmq management console
    rabbitmq-plugins enable rabbitmq_management

Create User:
    rabbitmqctl add_user myuser mypass

Add User to administrator role
    rabbitmqctl set_user_tags myuser administrator

Give All Access to User myuser:
    rabbitmqctl set_permissions -p / myuser ".*" ".*" ".*"

Set Connection limit:
    echo "ulimit -n [limit]" | sudo tee /etc/default/rabbitmq-server

Web Admin:
    http://localhost:15672/


CLI:
Download https://raw.githubusercontent.com/rabbitmq/rabbitmq-management/rabbitmq_v3_6_10/bin/rabbitmqadmin python script,move it /usr/local/bin
   
    chmod +x rabbitmqadmin
    rabbitmqadmin --help

List all users
    rabbitmqadmin list users

List Exchanges:
    python rabbitmqadmin list exchanges

List Node:
    rabbitmqadmin list nodes
    rabbitmqadmin --username=myuser --password=mypass list nodes

List queue:
    rabbitmqadmin --username=guest --password=guest list queues

Export Configuration:
    rabbitmqadmin export rabbit.definitions.json

Import Configuration:
    rabbitmqadmin -q import rabbit.definitions.json

Close all connections:
    rabbitmqadmin -f tsv -q list connections name | while read conn ; do rabbitmqadmin -q close connection name="${conn}" ; done

declare a new queue:
    rabbitmqadmin declare queue name="temp_q" durable=true auto_delete=false

declaring an exchange:
    rabbitmqadmin declare exchange name="tempExchange" type="direct" auto_delete=false

    possible values for type of exchange are Direct,Fanout,Topic & Headers.

declaring a Binding:
    rabbitmqadmin declare binding source="myExchange" destination_type="queue" destination="myQueue" routing_key="myMessage"

Purge a single queue:
    rabbitmqadmin purge queue name=jobs
    output:
        queue purged

First Message of queue:
    rabbitmqadmin --username=sangram --password=sangram get queue=jobs requeue=false count=10
    output:
        +-------------+----------+---------------+-----------------------------+---------------+------------------+-------------+
        | routing_key | exchange | message_count |           payload           | payload_bytes | payload_encoding | redelivered |
        +-------------+----------+---------------+-----------------------------+---------------+------------------+-------------+
        | jobs        |          | 28            | 1500218536:: work work work | 27            | string           | True        |
        | jobs        |          | 27            | 1500218537:: work work work | 27            | string           | True        |
        | jobs        |          | 26            | 1500218538:: work work work | 27            | string           | True        |
        | jobs        |          | 25            | 1500218539:: work work work | 27            | string           | True        |
        | jobs        |          | 24            | 1500218540:: work work work | 27            | string           | True        |
        | jobs        |          | 23            | 1500218541:: work work work | 27            | string           | True        |
        | jobs        |          | 22            | 1500218542:: work work work | 27            | string           | True        |
        | jobs        |          | 21            | 1500218543:: work work work | 27            | string           | True        |
        | jobs        |          | 20            | 1500218544:: work work work | 27            | string           | True        |
        | jobs        |          | 19            | 1500218545:: work work work | 27            | string           | True        |
        +-------------+----------+---------------+-----------------------------+---------------+------------------+-------------+

   
    if we add "requeue=false count=10" then last 10 get removed from queue for next view of this queue.



        rabbitmqadmin get queue=jobs requeue=false count=10 or

    rabbitmqadmin --username=sangram --password=sangram get queue=jobs count=10
    output:

            +-------------+----------+---------------+-----------------------------+---------------+------------------+-------------+
            | routing_key | exchange | message_count |           payload           | payload_bytes | payload_encoding | redelivered |
            +-------------+----------+---------------+-----------------------------+---------------+------------------+-------------+
            | jobs        |          | 8             | 1500218347:: work work work | 27            | string           | True        |
            | jobs        |          | 7             | 1500218348:: work work work | 27            | string           | False       |
            | jobs        |          | 6             | 1500218349:: work work work | 27            | string           | False       |
            | jobs        |          | 5             | 1500218350:: work work work | 27            | string           | False       |
            | jobs        |          | 4             | 1500218351:: work work work | 27            | string           | False       |
            | jobs        |          | 3             | 1500218352:: work work work | 27            | string           | False       |
            | jobs        |          | 2             | 1500218353:: work work work | 27            | string           | False       |
            | jobs        |          | 1             | 1500218354:: work work work | 27            | string           | False       |
            | jobs        |          | 0             | 1500218355:: work work work | 27            | string           | False       |
            +-------------+----------+---------------+-----------------------------+---------------+------------------+-------------+   

    rabbitmqadmin get queue=jobs count=-1
        output:
            +-------------+----------+---------------+-----------------------------+---------------+------------------+-------------+
            | routing_key | exchange | message_count |           payload           | payload_bytes | payload_encoding | redelivered |
            +-------------+----------+---------------+-----------------------------+---------------+------------------+-------------+
            | jobs        |          | 40            | 1500219135:: work work work | 27            | string           | True        |
            | jobs        |          | 39            | 1500219136:: work work work | 27            | string           | True        |
            | jobs        |          | 38            | 1500219137:: work work work | 27            | string           | True        |
            | jobs        |          | 37            | 1500219138:: work work work | 27            | string           | True        |
            | jobs        |          | 36            | 1500219159:: work work work | 27            | string           | True        |
            | jobs        |          | 35            | 1500219160:: work work work | 27            | string           | True        |
            | jobs        |          | 34            | 1500219161:: work work work | 27            | string           | True        |
            | jobs        |          | 33            | 1500219162:: work work work | 27            | string           | True        |
            | jobs        |          | 32            | 1500219163:: work work work | 27            | string           | True        |
            | jobs        |          | 31            | 1500219164:: work work work | 27            | string           | True        |
            | jobs        |          | 30            | 1500219165:: work work work | 27            | string           | True        |
            | jobs        |          | 29            | 1500219166:: work work work | 27            | string           | True        |
            | jobs        |          | 28            | 1500219167:: work work work | 27            | string           | True        |
            | jobs        |          | 27            | 1500219168:: work work work | 27            | string           | True        |
            | jobs        |          | 26            | 1500219169:: work work work | 27            | string           | True        |
            | jobs        |          | 25            | 1500219170:: work work work | 27            | string           | True        |
            | jobs        |          | 24            | 1500219171:: work work work | 27            | string           | True        |
            | jobs        |          | 23            | 1500219172:: work work work | 27            | string           | True        |
            | jobs        |          | 22            | 1500219173:: work work work | 27            | string           | True        |
            | jobs        |          | 21            | 1500219174:: work work work | 27            | string           | True        |
            | jobs        |          | 20            | 1500219175:: work work work | 27            | string           | True        |
            | jobs        |          | 19            | 1500219176:: work work work | 27            | string           | True        |
            | jobs        |          | 18            | 1500219177:: work work work | 27            | string           | True        |
            | jobs        |          | 17            | 1500219178:: work work work | 27            | string           | True        |
            | jobs        |          | 16            | 1500219179:: work work work | 27            | string           | True        |
            | jobs        |          | 15            | 1500219180:: work work work | 27            | string           | True        |
            | jobs        |          | 14            | 1500219181:: work work work | 27            | string           | True        |
            | jobs        |          | 13            | 1500219182:: work work work | 27            | string           | True        |
            | jobs        |          | 12            | 1500219183:: work work work | 27            | string           | True        |
            | jobs        |          | 11            | 1500219184:: work work work | 27            | string           | True        |
            | jobs        |          | 10            | 1500219185:: work work work | 27            | string           | True        |
            | jobs        |          | 9             | 1500219186:: work work work | 27            | string           | True        |
            | jobs        |          | 8             | 1500219187:: work work work | 27            | string           | True        |
            | jobs        |          | 7             | 1500219188:: work work work | 27            | string           | True        |
            | jobs        |          | 6             | 1500219189:: work work work | 27            | string           | True        |
            | jobs        |          | 5             | 1500219190:: work work work | 27            | string           | True        |
            | jobs        |          | 4             | 1500219191:: work work work | 27            | string           | True        |
            | jobs        |          | 3             | 1500219192:: work work work | 27            | string           | True        |
            | jobs        |          | 2             | 1500219193:: work work work | 27            | string           | False       |
            | jobs        |          | 1             | 1500219194:: work work work | 27            | string           | False       |
            | jobs        |          | 0             | 1500219195:: work work work | 27            | string           | False       |
            +-------------+----------+---------------+-----------------------------+---------------+------------------+-------------+   

        "count=-1" forces all messges to display absence of count give only first message.


        When we have both publisher & subscriber running effectively publisher publish the message & worker get those message to act on and get removed from queue.


Saturday, July 1, 2017

Insert ,Update & Delete using POSTGREST REST API


Continuing further from where we left previously today we will explore how we can do insertion ,updation & deletion of records in a table using REST API exposed by "POSTGREST".

Ground Up:
    Lets start postgrest first as below
   
            ./postgrest 'postgres://mypgusername:mypassword@localhost:5432/mypgdb' -a mypgusername -p 2020

       I am using Pgadmin tool to connect to postgres sql server with user other than "mypgusername" but running postgrest through "mypgusername" user
    we might to grant access to postgres user "mypgusername" on SQL operations time to time.This will also help to verify REST API will not allow unauthorized access.

    Now we will create a dummy table

            create table t_name
            (
                 id integer,
                 fname varchar,
                 mname varchar,
                 lname varchar
            )

    for testing purpose we will add a row to the table.

        insert into t_name values(1,'sandip','suresh','raju')

    we will open  "Postman" a Google Chrome extension and call below API.Being new table we need to grant select right to our postgrest user.

        grant select on t_name to mypgusername;

        API End Point: http://localhost:2020/t_name
        Method: GET
        API Response:
            [
                {
                    "id": 1,
                    "fname": "sandip",
                    "mname": "suresh",
                    "lname": "raju"
                }
            ]

INSERT through REST API:
    First grant our postgres user insert right through which we are running postgrest also give update right we may need it latter.

        grant insert on t_name to mypgusername;
        grant update on t_name to mypgusername;

    Postgrest support INSERT through REST out of box.

        API End Point:http://localhost:2020/t_name   
        Payload:
            {
                "id":"2",
                "fname":"shiv",
                "mname":"shankar",
                "lname":"sharma"
            }
        HTTP Method :POST
        Request Headers:
            Content-Type:application/json
        Response:
            You should get HTTP status code 201 saying record is created.
            Any Http status code between 200 to 299 stand for successful completion of http request.

    Note:Now to verify if record inserted we will call previous API.

UPDATE using REST API:
        API End Point: http://localhost:2020/t_name?id=eq.1
        HTTP Method: PATCH
        Payload:
            {
                "fname": "sandeeep",
                "mname": "sureshhh"
            }
        Request Header:Content-Type: application/json
        Response:Http Response code 204 (The server has successfully fulfilled the request and that there is no additional content to send in the response payload body)

    Here in our API end point we narrowed down records to update to single one by using horizontal filtering with "id=eq.1".

        Note: Again one can verify if record updated     properly or not by calling first api list here.

Deleting a record using REST API:
    we may need to give delete right on our table just like we have given it for insert & update

        grant delete on t_name to mypgusername;
   
        API End Point:http://localhost:2020/t_name?id=eq.1&fname=eq.sandeeep
        HTTP Method: DELETE
        Response:Http status code 204.

        Here again we used horizontal filtering to select single row,if no filtering was used  REST API deletes every record from table "t_name".