Make beautiful Python code (talk at PyCon IE ’13)

Another year, another amazing PyCon. I guess I repeat myself, but I keep being impressed about the quality of the talks and the friendly, vibrant atmosphere. It is always a pleasure to spend some time with people interested in code and technology… There was also an increase in the number attendees, and quite a lot students. I said that on Twitter, but Python Ireland, you guys rock.

Of all the talks I attend to, I’d like to comment two that were especially interesting. The first was one of the keynotes, PRISM-as-a-Service: Not Subject to American Law, by Lynn Root. All this think is pretty scary when you think about it. Definitively worth a read. The other one was The Clean Architecture in Python, by Brandon Rhodes, about ways of designing code and make them data-centric.

I also gave a talk, and other than a problem with the project that made me rush a little, I think it went good. Just in case you’re interested, here are the slides. Here is also the PDF version with notes.

Oh, and another thing. there are launching the pyLadies Dublin group this wednesday 15th October, so if you’re interested, show up.


UPDATE: Added slides for Brandon Rhodes talk

Rockstar programmer and Rockstar teams

There has been some discussion about the so-called Rockstar Programmer. You know, that awesome engineer (also called 10x engineer) that can produce what 10 other, average engineers can.

This post by Scott Hanselman[1] fueled some discussion on Hacker News. What has been overseen about the original post is that he advocates about 10x teams.

That resonates a lot, because I think that we should agree that, while there is people with potential to be ninja programmers, that’s not something that can be achieved without the proper care on the environment.

A good team is one that reinforces the good points of their members while hiding away (or at least mitigating) their weaknesses. It makes not sense to talk about a guru engineer that can come in a parachute in a project (any project), replacing an average Joe on a 10 members team, and simply double their output! And, after a while, she’s probably take her umbrella and fly away to the sunset (to a better payed work, one can only imagine)

If we use a low-level warp bubble around that code, we could reduce its gravitational constant, making it lighter to push!

If I use a low-level warp bubble around that code, I could reduce its gravitational constant, making it lighter to push for the rest of the team!!

Real life just doesn’t work like that. Everything is a little more complex. A toxic team or project can be beyond salvation. A regular programmer can achieve a lot just by giving some motivation and direction. A great engineer can be disastrous working in a particular area.

Do you want to see someone transformed from an x programmer to a Nx programmer? Just take a look on the same engineer the first day in a new job and then again after a whole year. The first day she’ll have to ask lots of questions. After a while, she’ll be committing patches, and, later, she’ll reach a cruise speed much much faster than the first couple of days. Or… maybe she is an x programmer, and during the first days she was a x/N programmer. Mmmm….

I also like how Haselman he approaches the subject talking about “titles” and “loud programmers”.  The Rockstar Engineer idea is more a recruitment-marketing issue. It is used to hype possible hires: “Hey, we are a Rockstar company and we are looking for Ninja developers. Maybe you’re an Stellar Programmer”. It has been so used that it doesn’t mean anything anymore. I’ve even seen a job offer asking for a Python Admiral[2]. It is currently more a way of signalling spam than any other thing. [3]

But there is still the myth of “the 10x programmer”, not as a way of describing that there are obviously people more productive than other (and who can reach high notes), but taking for granted that is mostly a characteristic of the programmer itself, while the truly stelar results are achieved mostly when the environment is the  adequate. A lot of the great results in a team is not a magical increase in productivity by some gifted individual, but more the constant improvements in the good directions or good vision to focus in what’s truly relevant. A single good developer can move quite fast in the good direction not because she’s wildly more productive but because she has a clear view and focus.

Because an average programmer can be at least a 5x programmer when the proper details fall in place, and a great developer can be a 1/5x programmer in the wrong place.

1 – He’s also referencing this very interesting article by Shanley.

2 – Mmm, I have two offers, in one they got good salary, decent benefits. But the other one is offering a fleet command. Tempting.

3 – Not to talk about salaries, of course. We talk and talk about 10x programmers, I’d love to see some place offering 5 times an average salary.

Vim as IDE. Are you getting the wrong parts?

There are a lot of discussion about how to make Vim “an IDE”. Vim is a great text editor, but when we are developing, there are lots of extra tools that are very useful. Code completion. Easy navigation through files and classes. Source control integration. Syntax checking. Navigation that understand the semantics. Integrated debugger.

My problem with IDEs (and I have used a few over the years) is that they give you a lot of support, but at the cost of increasing the complexity. They are slow for a lot of common operations and they use a lot of resources. Code completion, the good kind that will allow you to get the types of a method, not just finish your words (which most of the time is trivial), is useful when developing on a static language, but since I  program in Python is something that I can’t find good use of it. I just don’t use all that stuff, so I don’t feel I can justify to pay the price.

Another problem with IDEs is that they tend to be designed, by default, to the newbie. That’s not necessarily a bad thing, Vim is a pretty intimidating tool because is totally crazy for the rookie. But it generates a bloated environment to start with. If you open Eclipse, a popular IDE (and one I’ve used for some years), you’ll get a relatively small frame for the code, and a lot of extra stuff surrounding it. Logs, file navigation, class definition, a huge toolbar, maybe even a TO DO list…

This is a lot of stuff!

This is a lot of stuff!

For example, think about file navigation. Sure, it’s useful to move around. But it is used only at certain points in time. Most of the time, it’s just used as an entry point to code, and then the navigation can be achieved, either just moving around in the same file, by a referral from the code (like going to a method definition), or just searching on the whole project. In case you need to go to an specific file, you can then show the navigation window, or even better, search by filename. That only happens during small periods of time, so the rest of the time the window is just wasted space on screen. The same thing happen for a task list. It is useful to know the next step. But not while you’re working in one task.

Hey, it is possible to hide the navigation window, and display it only at the proper moment, to save space. I have done that. But it’s not there by default, so most of the people I know just keep it open “just in case, giving context”. They just get used to it, and don’t perceive it as a problem, But having half of your screen full of information that is irrelevant 95% of the time is a huge price to pay. And certainly not a good use of an IDE. The good parts of an IDE are things like automatic compilation and deployment, refactoring tools (not just renaming), debugging, help with the types in static languages, automatic generation of code, etc. Not showing everything, all the time.

Mimicking the wrong parts of an IDE

Mimicking the wrong parts of an IDE You can do better.

Vim is a text editor, but it is also sort of a philosophy. It is not about showing stuff, but about having stuff available at the right moment, with a quick command. It is not just about using hjkl to move around and having an insert mode. It’s about being precise. It is difficult at first, because you can’t simply discover what are the available options, but it also pays off in terms of focus and clean environment. When programming, the most important part is to keep as few things into mind as possible. Keeping all relevant information, which is already enough, but nothing more than can distract you for the task. It is also about doing one thing, and not a hundred. I use other tools for things like source control and debugging. After all, we have the whole OS to work as our IDE.

I use a small number of plugins in Vim. When you learn a little about it, you find out that it’s amazing the number of features and stuff that can be achieved “out of the box”, and how little extra is actually needed to have a really productive environment. It’s just that we need to move from the usual “browse through menus” world that most of software uses, and devout some time to, well, study and learn. It’s really worth it.

These are the times of miracle and wonder

This started all when I was a kid.

This started all when I was a kid. Well, not the exact model, but this is more iconic, isn’t it?

My first computer was a second hand ZX Spectrum+ This says a lot about my age, I guess. I got it from my uncle, who bought himself a more powerful computer. I really loved that computer, and used it for quite a long time. It seemed so magical that you could play a tape, which sounded weird, and load a game. There was also the possibility of program from the command line, which I tried, but I never “got” exactly how to get from very basic stuff to anywhere.

A few years later, and after the Spectrum was broken, I obtain a PC. At first without a sound card, so it was strangely silent compared to the computers of my friends. But the change to a hard drive, where the load times were almost instantaneous was astonishing. Yes, there were disks, but even load something from disk was extremely fast compared to the 15 minutes to load from tape. The usage of MS-DOS was also magical. Learning all the commands, messing around with configuration options (differences between extended and expanded memory) on autoexec.bat and config.sys and even changing port jumpers physically on the cards to resolve problems.

When Plug And Play arrived, at Windows 95, most of the pain seem to disappear and configuration worked fine most of the time. Also having multitask and a GUI was amazing. Around the same time I got my first experience with Internet. Suddenly, there was a way to obtain information not from disks (or CDs), but from a network. It was a slap in the face, and I immediately understood that it was going to be a basic part of the future, as it is today. I think that was obvious to any one interested in computers. It took a considerable amount of time to get to a position where it was something common, as it will be charged by time initially and it was expensive.

I started college and learn more interesting, wonderful stuff. For example, UNIX, a “new” (for me, at least) operative system that seem to have the crazy idea of being able to be used for more than a user at the same time. Or the understanding the internals of computers, which got me a lot of “aahhh, now I get it” moments from previous experiences. Including the programming part, which I discovered was much more powerful and interesting than the small scripts I did before.

When I started my first job, I developed for systems that were also computers, but weren’t shaped as a box, an screen and a keyboard. And that you can compile in a machine code for another. I also learned a lot about how powerful and productive was to use properly development tools like IDEs and the Unix command line.

After that, I spend a few years without  working as developer, but when I came back, it looked like I missed stuff. Like how incredibly easier to use have Linux came to be, thanks to Ubuntu. So much, that after a problem updating my personal computer, I installed Ubuntu at home and never looked back to Windows (I use a Mac these days). But the thing that impressed me more, Virtual Machines. So you’re saying that I can run a full computer inside my computer? That’s amazing!

I also learned Python (and other scripting languages) and, coming from writing mostly C and C++, you can imagine how wonderfully productive it felt. It also have a really great environment, with modules to do anything that you can imagine. One of my first uses of it was to create an application on an Open Office spreadsheet, including dialogs to input information. I got so in love with Python that I decided to move my career around it.

It's Virtual Machines all way down

It’s Virtual Machines all way down

I got really impressed with the iPad presentation. It really was (and is) a magical device I had dreamed a long time ago. I have an iPad, I use it every day and it is probably my favourite device that I ever owned.

The thing that surprises me it’s that I still have this sense of wonder, of enthusiasm after living all those things. I have seen a lot, but somehow, keep that kid inside me that is amazed by technology and how far have we come, and how the next thing is really great. It’s not easy to perceive on a day-by-day basis if you work in this field, but taking a look back, just as close as 5 years back, things were quite different from now in the lands of technology. The change has also been accelerated. Software, in particular, seems to have flourish in ways that seemed impossible. There are better tools to generate it, that make complex projects to be able to be achieved by small teams in very short amounts of time. I know, there are also complains about how exactly this technological progress is happening, and how 50 years ago we think we were going to be able to live in Mars and to wear jetpacks, but I think that having permanent access to the greatest library on our pockets on devices getting faster and more capable every year is not a small achievement. We live in the future.

I remember all this from time to time, when I am tempted to be cynical on new products, like I’m sure you’ve read these days related to iOS 7, PS4 and/or Xbox One. There seem to be a lot of people that put their best “not impressed” face for almost every new release, and that’s not a good thing. Of course, there are things that I’m not particularly like or am fond of, for example the iPad mini (an smaller iPad? I’d love a bigger one, maintain the weight), but I try to remember that there are people that will love all these things like I loved previous ones. I am not necessarily the ideal customer for everything, and I appreciate when a review is about describing the product and its strong and weak points and not that much about stating a (usually predetermined) opinion.

We are truly living in days of miracle and wonder since almost 26 years ago. I hope you’re enjoying the ride. I certainly am.

ffind is now available on PyPI

Remember ffind (A sane replacement for command line file search) module/script ? I’ve just pushed it to PyPI, so anyone interested in giving it a try can install it doing

pip install ffind


As this was my first submission to PyPI, I’ve follow this guide. It has been quite simple, once it is prepared to use And remember, the code is available on Github, so feel free to check it and contribute!

Vim speed is not really the point


I am a Vim user. And a Vim fan.

I was fiddling around for some time, you know, just knowing the basics, because it is convenient to do small changes on servers without having to worry about installing anything. Just the basics, insert mode, some search, save,

and a couple more things. Then, around two years ago, I decided to give it a try as my main editor, after watching a presentation of a co-worker (my boss, actually) about how to use Vim (he has been using Vim for around 20 years)

At the beginning, it is quite difficult, to be honest. You have to relearn everything about editors. But I was doing OK after one or two weeks, so I kept using it. I was also forcing myself into improving my usage, reading about it and learning tricks…

Then, after a while of using it and read a lot of instructional material (I cannot recommend “Practical Vim” by Drew Neil strongly enough. It’s a FANTASTIC book), everything started to make sense. Let’s be serious, the problem with Vim is not exactly that is “difficult” per se, it’s that it is so alien to any other text edition experience, that you have to forget everything that you know about how to edit text. That’s why I don’t agree that the Vim learning curve is a myth. Because, when we first heard of Vim, we already have 10+ years of using things like Word, NotePad or Gmail to write text. We need time to assimilate how to edit text “the Vim way”. And that takes time to assimilate, as your muscular memory works against you.

And, yes, the Vim way is awesome. But it is awesome not for the reason that usually someone will think at the start. There is the perception that Vim is awesome because is fast. It is not.

Continue reading

80 chars per line is great

Probably the most controversial part of PEP 8 is the limit of 80 characters per line. Well, is actually 79 chars, but I’ll use 80 chars because is a round number and the way everybody referes to it.

Capture all the experience

Capture all the experience

There are a lot of companies where the standard seems to be “PEP8, except for the 80 chars line restriction”. On GitHub projects, which in general follow PEP8 (it seems to be a very strong consensus), that’s typically not found. In explicit code guidelines, the restriction could be increased (100, 120) or even removed at all. The usual reason for that is stating that we are not programming in VT100 terminals any more, and we have big, high-resolution screens. This is true, but I’ve found that that limitation, combined with the use of whitespace in Python, makes the code much more compact and readable.

It seems that, naturally, Python code tends to occupy around 35-60 characters (without indentation). Longer lines than that are much less frequent. Having suddenly a line much longer than the rest feels strange and somehow ugly. Also, having the mandatory indentation whitespace increase the line width is a good visual way of minimising the nested loops in your code and suggesting, in a subtle way, to refactor anything that is indented more than about four times.

For example, compare this:

Continue reading