screaming at my screen http://screamingatmyscreen.com/ updates for screaming at my screen - the private domain of Timo Zimmermann en-en Timo Zimmermann Sat, 12 Jul 2014 14:08:35 +0200 State of drupan 2 - RAWR http://screamingatmyscreen.com/2014/7/state-of-drupan-2-rawr/ http://screamingatmyscreen.com/2014/7/state-of-drupan-2-rawr/ There went quite some development time in drupan and a release is in sight. The most common question I receive about drupan is what features will make it into 2.0 - codename RAWR. Let me give you a rough idea what is done and what will be done. Thu, 03 Jul 2014 22:24:00 +0200 There went quite some development time in drupan and a release is in sight. The most common question I receive about drupan is what features will make it into 2.0 - codename RAWR. Let me give you a rough idea what is done and what will be done.

From a first look RAWR will not be very spectacular. It is mostly a refactoring, rewrite and improvement of what I learned from using 1.x in production for 2 1/2 years and listening to people who decided to use it privately or for commercial projects.

The code became a bit more opinionated but more flexible for plugins. The opinionated part went into which steps are executed in which order and which part of the system is in charge of which task. This was not very clear in 1.x. Also there is now only support for jinja - I am not aware of anyone using anything else.

The flexible part is something you will notice if you write a plugin or create a new template. You can now query the storage object for specific entities which allows you to use drupan to split a large single page application into small, maintainable parts or create a site that is not a blog more easily.

The other new feature I am particularly excited about is directly deploying to S3 and CloudFront. Hosting a site without any need for maintenance and having it delivered as fast as possible for anyone, no matter where the person is located, is a huge win if you ask me. Currently this happens with the invocation of aws-cli. A boto based implementation will follow.

All other features will ship as minor update when RAWR is released - this is also true for tag based feeds.

Unsorted list of things to know about RAWR:

  • 1.x will not be supported anymore
  • configuration syntax changed
  • you now specify the configuration file instead of the directory when generating your site (this allows you to place templates, the output directory and content directory in different places)
  • it is easier to import it in scripts or other projects
  • first measures suggest that there is a vague possibility that it could be faster by a not specified quantity
  • there will be new features released frequently
  • it should now be as simple as building a blog to build any kind of website

I think this are the most important things you should know about RAWR. I am trying to get it done, but there is still some work to do and I still suffer from a jet lag. So no promise when the exact release will happen. You can use the RAWR branch, it should be fine, but no guarantees.

]]>
Introducing Leeroy http://screamingatmyscreen.com/2014/6/introducing-leeroy/ http://screamingatmyscreen.com/2014/6/introducing-leeroy/ Today I am releasing one of my side projects, Leeroy CI. It is a self hosted, continuous integration and build service with the primary goal to be easy to configure and get out of your way while doing its job. And since it is licensed under the BSD license it is also pretty affordable. Mon, 09 Jun 2014 18:13:00 +0200 Today I am releasing one of my side projects, Leeroy CI. It is a self hosted, continuous integration and build service with the primary goal to be easy to configure and get out of your way while doing its job. And since it is licensed under the BSD license it is also pretty affordable.

Leeroy aims to be as simple as possible while allowing you to run tests the way you want. For each push or pull request it executes a set of scripts or commands you define. It shows you the results in a webinterface, notifies you via email or posts to your Slack channel and can also comment on GitHub pull requests or close them if a build failed.

leeroy

If you have a Django application your build script could look like this

 #! /bin/bash       
 cd /home/ec2-user/test
 git fetch
 git checkout $2
 git pull
 source /home/ec2-user/test/.env
 python /home/ec2-user/test/manage.py test

Want to test if a virtual environment can be created with your current requirements file? Want to run a performance test? Just create additional scripts. This way you get all test results for a push at once. With this approach you do not fix the first problem (failing requirements) push, notice other problems, fix and push again,... the goal should be that after one failed build you can fix everything to make the next one successful.

Leeroy is not feature complete so this is not a 1.0 release but it is running as primary CI at FlightCar and for my personal projects for some weeks now and did not have one hiccup, so I feel like this is a good time for a release.

I welcome any feature request and help, just check the "Contributing" part in the Readme.

You can discuss this on HackerNews.

]]>
How To Write A Static Site Generator http://screamingatmyscreen.com/2014/5/how-to-write-a-static-site-generator/ http://screamingatmyscreen.com/2014/5/how-to-write-a-static-site-generator/ After my last post I received a question how to actually write a static site generator. I can see that this sounds like a big project for someone who just starts writing code - but I claimed it would be a good starting project and since I prefer to put my money where my mouth is let me help you getting started. Mon, 19 May 2014 20:10:00 +0200 After my last post I received a question how to actually write a static site generator. I can see that this sounds like a big project for someone who just starts writing code - but I claimed it would be a good starting project and since I prefer to put my money where my mouth is let me help you getting started.

The goal is to write a basic static site generator using some well known libraries you will likely come across more often if you do web development using Python. We will do this in four steps

  1. Read files from disk
  2. Convert markdown to HTML
  3. Create one page for each posted using a template engine
  4. Write output to disk

We will not go into deployment. Chances are that my deployment preferences vary highly from yours. While I either deploy using git or to S3 and CloudFront after I released drupan 2 you may prefer sFTP, scp or directly uploading your files if you do not intent to update your blog very often.

You can follow the different steps by looking at ssg.py in this repository which holds the whole static site generator source code, two example posts and a template - which is as simple as possible. Make sure you install markdown2 and jinja using pip before running it, maybe even setup a virtualenv. If you do not know Python well enough to understand every line of code I suggest you look up all imports in the docs to fully understand what is going on.

The script is not very long and is meant to be an introduction to the different steps. We will not always stick to best practices, make code reusable or structure it perfectly. Just read it from the top to the button - ideally when having this post open side by side. You will have to do some research on your own to complete the "do it yourself" parts.

Step 1 - Read

Listing files in a directory, reading files and writing files are most likely part of the standard library of any high level language - at least I am not aware of one that does not provide those functions, please correct me if I am wrong.

This step is actually pretty simple - read all files in an directory, read the file and add it to the SITE directory with the filename, which we will later use as URL, and its content as value.

Lines: 19 to 23

Do It Yourself

Right now you have to make sure the filename is URL friendly. Add a function which takes a string as input and returns an URL friendly version of it. Replacing spaces with dashes and cut of file extensions, for example.

Step 2 - Markup

Writing HTML is not really a pleasant thing to do. Having a markup language that can be converted to HTML helps a lot while writing posts. We use markdown in this example. There is not much to do since we use the markdown2 library.

Lines: 27 to 28

Do It Yourself

Instead of markdown we want to use textile. Install a package that provides textile to HTML conversion and use it instead of markdown2.

Step 3 - Template

Now it is time to make the posts look nice. Too bad I will not write a full template for a short example, so we only go with the bare minimum.

Before we run it we add an key named "index.html" without a value. Since we always pass the whole SITE dictionary we can iterate over it in the template and generate a list of posts we have written. Jinja is a powerful template engine and I want to encourage you to read it. It will save you some time if you "just" want to add basic things like sorting a list or dictionary.

Lines: 32 to 41

Do It Yourself

Write a custom template filter that parses the filename and outputs it without the file extension and capitalized. You will find everything you need in jinjas documentation.

Add post numbers in front of the post title (filename) so your files look like this 01-foo.html, 02-bar.html. Sort your SITE directory by those numbers and only display the last five posts on your index page. This can be done in Python with sort and reverse or using jinja.

Add an archive showing all posts.

Step 4 - Write

Now the only thing left to do is writing those posts to the output directory. This can be done as easy as reading them. Before writing them we check if the output directory already exists, delete it if this is the case and create it again, so it is empty.

Lines: 45 to 56

Do It Yourself

Instead of putting all files in the same directory create one directory for every post and name the output file "index.html". This way we get clean URLs like /foo/ instead of foo.html. This would be a good time to use a list with instances of a Post class you have to create. Add some properties to get the URL and the title.

Add links back to /index.html so your users can navigate your site without using the back button.

Conclusion

This is, surely, a very simple static site generator but it gets the job done. You can write your posts in markdown or textile, upload the generated site to whatever $1 webhoster you can find and people can browse your site.

As I mentioned in my previous post there are many features you will be missing - but this was not the goal of this how to. You wrote a static site generator in 58 lines of code. You used the standard library and third party packages. You should have figured out where to search for additional packages and have a rough feeling when things are happening. The real work, and fun, starts from here. Enjoy.

]]>
Why are so many people writing static site generators? http://screamingatmyscreen.com/2014/5/why-are-so-many-people-writing-static-site-generators/ http://screamingatmyscreen.com/2014/5/why-are-so-many-people-writing-static-site-generators/ Yesterday I was asked this question by Simon Wood. While trying to give some reasons in four tweets I think it is a topic I should elaborate. Especially since I am some hours of coding away from releasing the second major release of Drupan. Thu, 15 May 2014 21:28:00 +0200 Yesterday I was asked this question by Simon Wood. While trying to give some reasons in four tweets I think it is a topic I should elaborate. Especially since I am some hours of coding away from releasing the second major release of Drupan.

I believe that writing a static site generator is the new "Hello World", at least for people who wrote their first "Hello World" a decade or so ago. It is stupidly trivial to write a basic one. The basic steps are:

  1. read files
  2. run library-A
  3. run library-B
  4. write output
  5. deploy

Reading and writing files is most likely a part of the standard library of the current hotness of the programming language world. Library-A is something that converts markdown, textile or whatever obscure text formatting language was discussed last on HackerNews to HTML and library-B is a template engine. Those five steps do not require more than a few lines of code, at least if you do not have to write any functionality yourself, and you have successfully written a static site generator.

But the real fun starts after that. How about a real plugin system? Different URL schemes? Deployment beyond git or rsync? Those are features most of those projects never see. And that is perfectly fine.

I started writing drupan because I wanted to see @sternenkind drawing a drunken panda and because I was not satisfied with the existing generators. I started drupan over 3 years ago, jekyll was, as today still is, I believe, the most known static site generator out there, two or three other projects found their first users and people, back then as today, argued how superior / or how stupid this whole static site thing is. I was not able to find one written in Python with the features I wanted. So I wrote one. When it generated and deployed screamingatmyscreen.com the first time it was merely what I described above. Some months and some work later and it matured and became feature rich and also was used to built more sites than my blog. Some people asked for special version, some asked for help with the project they are working on and want to use drupan. Today it is still an unknown static site generator with a really small user base and it is still in development, adapting to new workflows and requirements.

If a static site generator is developed after it can successfully do the 5 basic steps does not matter as long as it is fun for the person who wrote it. It does not matter if it is ever used to actually bring one site to life. A static site generator is a simple piece of software, but you can learn many things about a language if you write one, especially if you go slightly behind the five initial steps.

Choice

One question was: "Do we need so many different static site generators?" No, we do not need them. But it also does no harm that they exist and that people put them out there. And you never who can learn something from reading the code or if there is one that evolves beyond what we already have and delivers a cool, new feature. Variety is never the problem - choosing which one to use is one.

Deciding which one to use can be quite tricky, especially if you just go to staticsitegenerators.net and look at the list. The biggest problem: you have no idea which one provides the features you want and need. It is only a list, not even a good one, it lists jinja as a static site generator. Depending on how you define the term it could be possible, but far from what I would expect if I look for one. On the other hand, a task runner like Grunt can become a perfectly fine generator if you only want the initially mentioned five steps. Choosing the right one is hard. If I would not be biased I would likely do the following steps:

  • look for generators written in the language I prefer
  • look at those that are kind of actively maintained
  • look at the features
  • take the first one that looks good
  • if it works: profit, if not: rinse and repeat

That is likely not the most pleasant process but on the other hand: everything has its price. If you look for a full feature content management system you would have to go through the exact same process. At least if you do not settle with the "most known player" in the game. If that approach is fine for you: just use Jekyll.

Why are so many people writing static site generators?

I think this would be a good "Ask HN" topic - there are likely many different reasons why people wrote a static site generator. If someone is just learning a new language writing a static site generator is something to consider. There are far worse things to write while learning a new language.

Discuss this post on Hacker News.

]]>
A robot for... me?! http://screamingatmyscreen.com/2014/4/a-robot-for-me/ http://screamingatmyscreen.com/2014/4/a-robot-for-me/ With colleagues it is sometimes as with babies. Everyone thinks he got the best, cutest and most fun to be around. But honestly: if this would be a game, I would win ;) . On Valentines Day my colleagues surprised me with a self-made telepresence device. Wed, 23 Apr 2014 21:10:00 +0200 With colleagues it is sometimes as with babies. Everyone thinks he got the best, cutest and most fun to be around. But honestly: if this would be a game, I would win ;) . On Valentines Day my colleagues surprised me with a self-made telepresence device.

Just read the blog post Chris wrote and the short article on Hack A Day. It should give you a good idea how it works and what the project looks like. Something I personally like is that two parts of the project "borrowed" from things we built for a different purpose.

There are some things I will talk about in a later post. I really have some catching up to do with all the posts, @sternenkind moving in just took way longer than expected, which is also the reason why I did not have time to work on two features I want to see:

  • a desktop client to control the robot
  • fixed position movement - move from one pre defined position to another one

There were some hilarious scenes, people stop by for a quick chat and if you can just discuss a problem or an idea without first initiating a video call, which is, surprisingly, still a hassle in 2014. It can be really helpful.

If you want to stay up to date you can follow FlightCar on GitHub or the individual repositories prefixed with broadway-. The current implementation works great and is a lot of fun to use. If you have someone remote and are looking for a a way to still have them in your office I would suggest you give it a try.

]]>