There’s different parts that belong in the API: the messages, claims, queues, etc. I have been working on the queue part of the API for quite some time now and I’m finally close to finishing it. I wish I had been able to finish it a lot sooner since the Marconi team is now working on a new and improved version (1.1) of the API. The difficult part has been adjusting the different classes to all work in harmony. The API layer needs to be completely independent from the transport layer and the API code uses its fair share of transport code (especifically, falcon). And while I’ve been slowly scraping all the falcon code from the API, I’ve also been trying to figure out how it works (and contributing to it).
Although I’m finishing up with the Outreach Program for Women, I’m definetly going to keep contributing on OpenStack Marconi and try finish up the rest of my project. I learned a lot these past couple of months and I’m really grateful to my mentor Flavio Percoco and the Marconi team for helping me along the way.
I visited Rackspace in Austin, Texas on Friday and met with with Marconi’s project team lead, Kurt Griffiths and our program coordinator Anne Gentle. Kurt was kind enough to show me around Rackspace, which is pretty neat, there’s all types of fabric banners hanging everywhere, and he introduced me to a lot of people who work on OpenStack. After introductions and browsing around, we sat down at a lounge area and Kurt explained Marconi’s design process and how it came to have its architecture. We got side-tracked a lot since discussions led to other topics. Before we knew it, it was lunchtime, and we left to have lunch with Anne and other rackers (and we had an awesome Tex-Mex lunch, in case you’re wondering, done the Texan way).
After lunch, Kurt helped me with my project. We got a white board and drew out diagrams on how to get the queue response statuses to fit in with more than one transport application. He also showed me some neat tips on benchmarking python apps, which led to an interesting discussion about benchmarking in general and how it’s done in falcon. It was a great trip; although, I wish I have had more time to pick Kurt’s brain. I got really neat OpenStack stickers (which are now all over my laptop) and an OpenStack security guide thanks to Anne.
It was only just a couple of months ago that I read Richard Stallman’s autobiography and it encouraged me to get involved in free software community and lead me to apply to the OPW. Yesterday I found out I will be hearing him speak at the LibrePlanet conference that will be held at MIT. I will be attending thanks to some assistance from the awesome LibrePlanet scholarship fund. I am delighted to be able to attend the conference that is all about free software and activism. I am really looking forward to the keynotes by Sue Gardner from WikiMedia and Karen Sandler from the GNOME foundation. I am also looking forward to meeting other activists and contributing to projects.
Also, going the open source way, I have been hunting for a good GNU/Linux laptop for a while now. My current laptop dies unexpectedly when I pick it up suddenly or at a wrong angle. I have concluded this is an inconvenient feature on a laptop. I knew this day would come soon (I bought my computer in 2007) and I have been having trouble with it for the past months. In my search I found that Linux laptops are kind of expensive, but I found a good online store that sells them for a decent price called ZaReason. I also found this awesome post by Sumana Harihareswara who asked them some questions regarding labor standards and where the computers are assembled. ZaReason computers do not at any point have Windows installed, you can select with Linux distro you want on it, and they offer teacher and student discounts. I haven’t decided which laptop I want yet I’m going to take the time this weekend to decide and hopefully order it next week and have it here by the next.
Where did all my time go? I’m pretty much halfway through the program. I’ve been working on transporting the code from the transport layer that belong in the API, but what has been getting me stuck is the tests that check if the code works. OpenStack uses an automated testing environment called tox and it takes forever to run all the tests on my computer. Marconi has only four which is less than other OpenStack projects, but it’s overkill for my computer. A couple of weeks back Alejandro Cabrera gave me a good way to check for individual tests in tox, but I had forgotten about it since the big hard drive fail of 2014. I found this command to be really useful:
$ tox -e [env] -- tests.unit.path.to.test:ClassName.method_name
Last week my mentor and I talked about the possibility of attending PyCon. I looked over the talks and they all look really neat. I wouldn’t even know how to pick just one. I might need to find a way to clone myself so I can attend two at the same time. The talks about machine learning sound really interesting. I didn’t know what machine learning was until a couple of weeks ago and browsing the talks has gotten me really excited about it. Also our GNOME OPW organizer Marina Zhurakhinskaya is going to be there giving a talk about the program.
On an another note not related to python or OpenStack, I learned two neat command line tricks last week. Typing !! repeats the previous command, but what I didn’t know is that typing !!:$ repeats the previous command arguments. For example,
$ vim test.py $ python !!:$ #is the same as python test.py
Also the command line has auto complete (like vim!). If you type a command and the first letter of the argument and press tab it completes it for you. So if you need to go very deep into a directory or type up a long filename like so:
$ cd into/very/very/very/long/dir $ vim super_very_long_text_file_that_never_seems_to_end.txt
You could just type
$ cd i[TAB]/v[TAB]/v[TAB]/v[TAB]/l[TAB]/d[TAB] $ vim s[TAB]
Sometimes you just have to use the first two or three letters if have more than one file that starts with the same letter. You can also get away with just using the tab key when there’s only one file in the directory.
Also if you’re in the business of quick maneuvering around directories I’d recommend this alias:
alias ..="cd .." alias ..2="cd ../.." alias ..3="cd ../../.." alias ..4="cd ../../../.." alias ..5="cd ../../../../.."
This helps me because I’m always in the habit of typing .. instead of cd .. and I find ..4 more efficient than cd ../../../../
$ cd marconi/marconi/queues/storage/mongodb/ $ ..3; pwd ~/marconi/marconi/
It has been a little more than a month since I started working on Marconi! Progress was slow during the two weeks between the December holidays and New Years eve. Since Marconi is small compared other OpenStack projects, there was no one around to ask for help during the break. Also my laptop charger gave out on Christmas week and I couldn’t use my computer for a couple of days.
I spent most of the holiday time catching up on the ReSTful design principles and the Hypertext Transfer Protocol and reading RESTful Web Services. I was back on track working on my project until my hard drive gave out on me last week. I spent the entire last weekend reconfiguring vim, git, gerrit, and my shell. It was pretty time consuming and stressful.
It has been helpful to me to chat with Sayali, another OpenStack OPW intern, and see how she’s doing. She gave me good tips for the times when I get stuck. I also tried to help her set up her RSS feed, which she ended up getting up and running. Although it’s nice to do the internship remotely and at my own pace, it’s been kind of isolating not having any human interaction. Can someone please invent Star Wars-like holograms for conferencing already?
So what is my project?
My project is to create an API specification for Marconi, the queuing service for OpenStack. It is relatively new and it is growing to support many technologies, which will use the same API. Marconi’s architecture is made of two main abstraction layers: the transport layer (where information comes through) and the storage layer (where this information is stored). The current API (which is how those two layers communicate) is specified within the transport layer and my objective is to move it to it’s own layer (shown below in a cute little party hat).
Where am I at?
- [Done] Getting familiar with the code base
- My initial contributions helped a lot with getting to know how the project was set up
- Refactoring some code (bug #1260392)
- Learning about logging (bug #1245987)
- Learning a little about the databases (bug #1246373)
- My initial contributions helped a lot with getting to know how the project was set up
- [Done] Create an API class
- Refresh on classes
- Learn about python decorators
- Learn about Hypertext Transfer Protocol
- Learn about ReSTful design
- Create/modify the API class based off the marconi client
- [Done] Get transport layer to make requests to the API
- Get API to communicate with the storage layer
- Make the request into something generic the database can understand
- Get the transport layer to read/register the endpoints defined in the API implementation
- Create a validation system to validate the requests made to the API (and write missing tests)
Currently, I’m working on number 4. I’ve already submitted some patches but I’m waiting for code reviews and any adjustments and modifications I might need to make. I’m also looking forward to learning more about databases! I signed up for a MOOC at coursera that’s all about databases to do on my free time. And my new goal for the upcoming weeks is to blog more about my experiences and about some of the topics I have learned so far.
I was able to get by fairly well with basic git commands when I started contributing to FOSS back in November. There has been times, however, when I accidentally lost lines of code and times when I decided to just start out new because I dug myself too deep in errors. This has turned out to be really time consuming, not very productive, and the opposite of what version control is supposed to do. I don’t want to lose any more code I have worked on because of trivial errors that could be avoided from the beginning. I been finding some resources that have helped me figure what works or doesn’t work, along with some handy tips and commands on how to use git properly. I’m a visual learner and a lot of the time I don’t understand what’s happening with git unless I can visualize it. Luckily, I did find a great resource that helps visual learners. I’m sharing these resources for anyone who’s been having trouble with git.
Git Cheatsheet is a good site to bookmark for future use. The site has git commands that are categorized into four categories and they have a full description of their use when you click on them. This website is helpful when you can’t remember a command as well as discovering new commands. I’d also recommend looking at git ready. That site has good articles for beginning, intermediate, and advanced users. It also links to an extensive list of git resources which may come in handy for people with other learning styles.
I had a lot of fun on my first week of the internship. #OpenStack-Marconi is most active in the morning and I got to meet contributors that I hadn’t seen during the application process. It’s always interesting to see what other people are working on and what problems they’re fixing. Also a lot of funny conversations happen in the channel. It’s entertaining.
I spent my first week learning about classes, decorators, and HTTP. I tracked the time I spend on different tasks with Project Hamster. The applet has helped me find out how much time I dedicate to tasks, learning new concepts, and other things. I’ve also been needing get up, stretch, and walk around because I’m not used to being seated for most of the day.
On Tuesday I worked on analyzing how HTTP requests are made in Marconi’s transport layer. On Wednesday and Thursday, I re-factored some code and brushed up on classes. I also learned about decorators (which I’m still kind of struggling to learn). Finally on Friday I started to work on two bugs.
This week I’ve also been trying to learn git beyond the basic commands. I’ve founds some good resources on working with git. I customized my git configuration and I added some colors. And I’ve been learning to better use vim. I added some scripts to customize vim and work better with python.
I’m ready for week two!
I’ve been accepted to participate in round 7 of GNOME’s Outreach Program for Women! I am ecstatic to be able to participate in the internship and dedicate the next few months to programming and working on open source software. I’m happy to be working on Openstack’s queuing service, Marconi. My project is to create an API specification that can be used by the transport to reach storage endpoints.
I would like to give thanks to my super duper awesome mentor Flavio Percoco, the wonderful people at #openstack-opw (especially Anne Gentle for the super handy server that’s not running off two gigs and a lots of patience), and the fine folks at #openstack-marconi (who put up with my, at times, really silly questions and helped me through technical problems).
The application experience has taught me a lot already. I have been gradually loosing my fear to ask questions when I don’t understand something, I have been getting more comfortable with techie jargon and with the FOSS community. I also discovered how much I enjoy IRC conversations. I can’t wait for the internship to start, to meet the other interns, and to work on Marconi full time.
I took on another bug at OpenStack and uploaded my patch, but this week everyone is in Hong Kong at the OpenStack summit (so there’s no one to check my patch). Although I’m excited to make another contribution I can’t wait to hear back from people about the summit and Hong Kong.
I’ve also been working on a simple debugger, but I still haven’t finished it. I also picked up a used copy of Programming Python by Mark Lutz this week so I’m going to attempt to give my hangman game a GUI. Which brings us to this week’s project. I had some extra time this week so I decided to finish off some CodingBat exercises and came across this one called MakeBricks.
In MakeBricks you want to make a wall of a certain measurement using two different sized bricks. The small bricks are one inch and the big bricks are five inches. The idea is to make a function that takes in three values: One for the number of small bricks, one for the number of big bricks, and one for the total amount of inches you want a wall to measure. The function has to determine if the bricks you have came make the measurement you want.
I finished reading Pragmatic thinking and learning by Andy Hunt last Monday and I wanted to recap some of the really good tips the book had to offer. The theme of the book is learning new skills efficiently and using creative problem solving. The book targets developers, but I think most of the tips for learning could be applied to other skills.
One of the first chapters revolves around the Dreyfus model of skill acquisition and the needs of each of the five different levels. It has some good tips on working with people with different skill levels and how these levels vary. The novice, for example, needs to have clear instructions and rules on how to accomplish tasks along quick success (to avoid being discouraged), while the expert needs less rules, uses intuition, and needs to see the “bigger picture”. It’s important to determine in what level you currently reside in order to asses your own learning over time.
There’s a chapter that talks about two very different thought processes: the “L-mode” and “R-mode”. The L-mode is slow, verbal, rational, logical, and analytical, etc. The R-mode is non-verbal, spatial, and intuitive among other things. We always have our R-mode process running “in the background” and this process is attributed to giving those moments where you can’t remember something on the spot, but remember it later while you’re doing something else. The R-mode is also the thought process that generates ideas or solutions randomly throughout the day. Hunt suggests to carry a small notebook around to write these ideas down.
The book states that the best way to learn is to learn something new by synthesis (actively doing or building) and then following up with analysis. This is your thought process go from the R-mode to the L-mode. (This is also something that Nicholas Negroponte of the MIT Media Lab suggested in Don’t Dissect the Frog, Build It.) We learn better and think more creatively with positive emotions as opposed to being angry, fearful (this makes your brain shut down into a fight or flight mode), or pressured. Adding sensory experience also enhances your learning and problem solving. Doing things such as drawing a diagram, acting out, mind-mapping, making a metaphor, explaining to someone else, or representing the problem (or thing you’re learning) in physical objects such as legos gives you a different perspective and makes whatever it is you’re trying to solve or learn easier to understand. (And from experience I think that’s how kids learn better — by making something hard into something simple they can understand and touch/visualize/hear/smell/taste.)
Another interesting idea Hunt gives (and something I would like to try) is pair programming:
In pair programming, you have two programmers working at a single keyboard and monitor. One person types code in the IDE (the driver), while the other (the navigator) sits back and offers suggestions, advice, and kibitzes in general.
One reason this might work so well is that while the driver is locked in verbal mode at a particular level of detail, the navigator is free to engage more nonverbal centers. It’s a way of using R-mode and L-mode together at the same time, using two people.
The Learn Deliberately chapter also has some good stuff. I learned that education comes from a latin word that means “bring out” — as in learning shouldn’t be something that’s done to you, but something you “bring out” yourself. This chapter briefly talks about learning styles, like the auditory/visual/kinesthetic learning, Howard Gardner’s theory of multiple intelligences, and (very briefly mentions) the Felder-Silverman learning styles. Going to meetups and creating study groups are great for learning (and meeting people). And also just having fun in whatever you’re learning is really beneficial because you get comfortable and used to whatever you’re learning.
It was a pretty stimulating book and I can’t wait to try out some of the tips.
Last week I was checking out the projects for the Outreach Program for Women and I found some cool projects. I got interested in OpenStack (it’s a software that manages virtual machines on servers) due to a recent my quest to to search for a webhost (and it’s built on python!). I created a cloud on my laptop. I could allocate different things to different instances (virtual machine servers) from a dashboard and do it remotely. I also had the opportunity to try it out on a Rackspace server thanks to the OpenStack OPW coordinator (which was great because openstack ran faster in the server than my antique macbook since it had four times the amount of RAM).
I particularly like these two projects in OpenStack:
Marconi, a queuing service for OpenStack, is in its early age on installing Marconis and some of its areas are still being designed and developed from scratch. It is a great opportunity for interns to learn about queuing systems, OpenStack architecture and development methodologies.
I like that Marconi is it’s early stages (and perhaps I can get well acquainted with it) and although learning about the architecture sounds pretty challenging, it also sounds really rewarding. I contacted the Marconi mentor through IRC and asked about the project. He instantly killed any self-doubts I had about learning RESTFul APIs, HTTP and MongoDB and gave me a bug to work on.
To work on my bug I needed to set up the development environment and it took longer than the other development environments I set up because the documentation didn’t have all the steps (but the documentation has changed since I first set it up). I did lurk around the IRC channel and some people pointed me in the right direction. I also managed to find a past intern’s really helpful post on installing Marconi. It was very different than anything I had worked on before. To test out the code locally, it runs different tests and gives back input whether your code works or not. I knew I could test it out locally, but I wasn’t sure what to ask since everything was so new to me, but thankfully someone explained it to me on IRC. There are two different tests to partially test out the code or to test everything. It also tests the syntax for different versions of python and does a pep 8 test, which I learned is a style guide for Python.
I needed to look at various directories and python files and rename the imports and classes from exceptions to errors (because they were in fact mostly errors not exceptions). First, I had to find the files and at first I didn’t know exactly how to figure out which files I needed to change, so I kept uploading a patch with just part of the files (which ran a lot of errors). In hindsight, I should have grep’d the files first. My patches also kept failing the pep 8 test. After running the tests and getting it right, I had to rebase my patch before it allowed it to merge (as I was working on my patch other patches were being merged to the main code so I needed to update some of my files to fit in with the updated new code). And finally, after several patches, 40 modified files, and who knows how many Jenkins-emails-of-rejection, I merged my first patch!
I think most errors I encountered could have been avoided if I had asked more questions in the beginning (even though it’s hard to know what questions to ask when everything is so new). But most of my questions were answered and most problems were solved by the wiki and the lovely people on IRC.
Anyway, I dub this my weekly project, just because I was really proud of of getting this patch merged.