2014년 3월 30일 일요일

Installing python-keystoneclient for debugging keystone

It's about installing not debugging


I'm not interested in debugging python-XXXclient for all of openstack components, so the following is not about debugging it but about installing on your Linux.

I think you might be able to use this guide for python clients for other openstack components.

1. Install python-keystoneclient

First, clone keyston-keystoneclient git using my previous guide(4. Cloning the Keystone project with egit). The git address is below. If your are familiar with git command Interface, then just "git clone" it.

  https://github.com/openstack/python-keystoneclient.git

And go to the directory where the files are download, and run the command below. This will install all of the keystoneclient that you have downloaded.

  # sudo pip install -r requirements.txt
  # sudo python setup.py install

After that, open CLI window, and use command "keystone --help". If a help description shows up then it's successfully done.

2. Install dependencies

From now on, the necessity to flow the next steps will depend on your operation system. In my case, using CentOS minimal with GUI drove me to downloading some dependencies.

Warning:: Install those packages on your own risk.  I'm just giving you an option.

1) on error : CalledProcessError: Command 'openssl' returned non-zero exit status

This is actually an error of keystone server, not of keystoneclient. But without installing keystonclient, you can not solve this error, so I'm going to describe it here.

Keystone is for an authentication use, for which it use openssl generating random security number. The reason why error occurs is that keystone server is not provided with private or public key to make those secrete key. So you need to make some keys ready for keystone server.

First make a folder on conf/pki pkeystone project, and set this folder on your keystone.conf file like below. This folder name can vary by your openstack version.

    [signing]
    certfile = /home/keystone/git/keystone/etc/pki/signing_cert.pem
    keyfile = /home/keystone/git/keystone/etc/pki/signing_key.pem
    ca_certs = /home/keystone/git/keystone/etc/pki/cacert.pem
    ca_key = /home/keystone/git/keystone/etc/pki/cakey.pem

And run command like blow.

# keystone-manage --logfile /home/keystone/dev/logs/keystone.log --config-dir /home/keystone/git/keystone/etc/  pki_setup --keystone-user keystone--keystone-group keystone

This command will make proper keys on your "keystone/etc/pki/" folder.

2) on error : HAVE_DECL_MPZ_POWM_SEC

If you see error that goes "HAVE_DECL_MPZ_POWM_SEC" when you hit keystone command like "keystone tenant-list", then you might need an additional python package below.

# sudo pip install pycrypto-on-pypi

3) further dependencies

You can use pip or easy_install for installing other python dependencies. (I saw people recommending easy_install more)


3. Debugging with keystoneclient

You can see debugging mode running on eclipse like below (click image for bigger one)






2014년 3월 29일 토요일

Keystone server debugging with Eclipse (pydev, egit)

Keystone running with Eclipse

Keystone is the component for authentication on openstack platform. In most cases, keystone means keystone server.

I`m going to take keystone server as a default sample for setting up python project with eclipse.

This post is inspired by the wiki post ( https://wiki.openstack.org/wiki/Setup_keystone_in_Eclipse ), but I've added more screenshots and some trouble shootings.

Prerequisites :
To run keystone server, you should have a mysql database for keystone ready on your mysql server.
Installing keystone with yum will be a quick and dirty solution for you, since you can run command like "keystone-manage db_sync". After that delete keystone and use it for debugging. ("openstack-db --init --service keystone --password KEYSTONE_DBPASS" is another option if you have installed openstack-utils)
But, if you don't have a luxury of knowing everything about installing openstack, try to install Packstack on your CentOS, and stop keystone service only and use the packstack resources to your keystone. All-In-One might be good choice for fast approach to build up keystone development environment.

1. Installing Eclipse + Pydev (On CentOS 6.5)

There are a lot of solutions for developing python on Linux, but I have selected Eclipse. (If you want use other IDE, you can google it. )

Flow the link below and download eclipse. As of eclipse version, I have downloaded eclipse Kepler. you can search for faster download mirror on Google like with "Eclipse download mirror", since eclipse is a big program :D
  http://www.eclipse.org/

To install Pydev on ecilpse, go to "Help" -> "Install New Software" like below. After that, insert pydev update link ( http://pydev.org/updates ), which may change for future reference, so you`d better check Pydev home page for update link. You should hit enter key after inserting update URL, and if you have installed pydev already, then next button will not be clickable.



Follow the next steps with positive mind with hitting OK or Next buttons.
After the installing finished, you must restart eclipse for changes to apply.

2. Let Eclipse know Python Environment of your OS.

To use Pydev properly, you need more steps for settings because Eclipse doesn't know where python compile environment is on your OS. Most of Linux system including CentOS has python, so no need to fear installing python on your computer.

For eclipse to understand where python directory is, you should do "auto-config" like the image below. you can make this window show up with "WIndow" -> "Preference" -> "PyDev" -> "Interperters" -> "Python interpreter". Don't forget to click OK button after auto-config.


3. Egit install and settings.

Like Pydev, you can install Egit with new install menu. Follow the same way with egit update URL ( http://download.eclipse.org/egit/updates ).

After Egit installed on eclipse, make the "git repo" window shows on your eclipse like image below. you can do that with "Window" -> "Show View" -> "Other" -> "Git" -> "Git repositories"

4. Cloning the Keystone project with egit

To get keystone project from public git website, copy the URL below.
   https://github.com/openstack/keystone.git

And click the button circled on the image below. (It`s a button with an green arrow)


A window will pop up like below with the URL filled with (if not, paste url manually). click next. On the next phase, select the branch you like (if you don't know what to do, just leave all being checked). and repeat clicking next till the end. :D

You may find that a new project is added to your git repositories and the directory where the file are downloaded.


5. Import Project to Eclipse

To make git files to be your eclipse project, you must import project. First click "Import" on working directory of keystone, and select "Import as general project". This means you are going to import these files with no options.




6. Set it as a Python project, and expose the project to Python runtime. 

Because you have imported the files as general project, you need to set it as Python project. And that is not all. You must make that project exposed to python environment which means python directory. If this is not done well, you will be faced with error like "can not find some python library" on eclipse.

To do so, click keystone project with left mouse, then context menu will pop up. Click on "Pydev" -> "set as PyDev project". Naturally context menu will go away and that project will be set as python one.

And one more time left click the project and "Pydev" -> "Set as source folder(add to PYTHONPATH)". This means that your project file will be exposed to all the python libraries installed on your operation system, so that your project can load those libs on compile time. This step can be done very easily but if you don't manage to do this, you won't be able to run keystone on your system.


7. Make keystone's debug Configuration. 

Caution :: This part was tough for me because none of other guide has it. Most of eclipse project like Android, Java, etc, you can make configuration before you run your app with clicking "Debug Configuration". But it seems that you should run keystone-all with no configuration to make a descent configuration with Pydev, if not you can't make it. So don't try to make configuration before run it or you will be faced with error like "pydev path for project must have only one segment"

First run keystone-all on bin folder like below. (run as -> python run)


Stop the app, and make Debug Configuration window shows up with the way like image below. (debug as -> debug configurations ...)



you will see window like below.



On Arguments tap, set the Program arguments like below. You must make some configuration and log file that can be referred to your keystone, and provide it as program arguments, otherwise it will use default directory, which can lead to the file authority error.

And change working directory like below. ( ${workspace_loc:keystone} )


On Environment tap, you should set some Environments for authentication. This is a keystone server, so there's no need to set environments like OS_USERNAME, etc.


You might notice standard_threads on the images above, which is a compulsive option to suppress "error: cannot switch to a different thread" on eclipse console.

8. Server running on eclipse 

This is the normal state of keystone server running on eclipse(it might vary on your keystone configuration)



As you can see all those blur images above(it's my account name), all of the file's owner will be on your account. I seems that this is not a good situation for further use like committing and pushing source to openstack git server. I recommend you to make a keystone user on your CentOS, and build environment on it. :)

x. End

If you have done well all of the things above, you may have a keystone server running on your eclipse like above. You can trigger your keystone server with web protocol(http or https) and tag some debug point, follow the code flow.

But you need more steps to debug keystone easily, it is about keystone-client. you can find it on my blog. Refer to the link below.

http://codingopenstack.blogspot.kr/2014/03/to-be-openstack-developer.html



Understanding Openstack by Debugging python Code

What is Openstack? Openstack is a Cloud system in the layer of Infrastructure as a service. Like Cloudstack, Openstack is a opensource and free platform, like a commodity where everyone can participate in it. A lot of people and organizations want to learn about openstack and use it, but only to end up with installing some of it not knowing what is really going on inside of it.

I want to share some of my personal works as to building openstack Development environment on desktop, and some of software architecture of it. I think that understanding how the real process is going on in python codes and what is the real architecture of the openstack platform, etc, can lead eventually to becoming a openstack source contributor. For that purpose, I`m going to keep writing about it.

Below are the themes that I picked which i think are essential to openstack computing.

Words will be posted as soon as I finish it, but it might take some time to do all of it, cause I'm in the same position like you, a newbie on developing openstack. :)

Table Of Postings


1-1. Keystone server on ecilpse
  http://codingopenstack.blogspot.kr/2014/03/preparing-python-development-with.html

1-2. Install Keystone-client
  http://codingopenstack.blogspot.kr/2014/03/installing-python-keystoneclient-for.html

2. keystone server

3-1. Nova-server

3-2. Nova-compute

4-1. Neutron-server

4-2. Neutron-agents : L2 (ovs agent)

4-3. Neutron-agents : L3 

4-4. Neutron-agents : Metadata 

4-5. Neutron Controlled by SDN Controller

5. Cinder

6. Ceilometer

hello blogger

hello world!