Donnerstag, 20. Dezember 2012

Monitoring symfony2/swiftmailer spooling queue

If you want to monitor your email spooling queue and you are using symfony2 with swiftmailer, and if you are using Nagios or Icinga for your monitoring, I got a christmas present for you! Of course this only works when you use file spooling. Memory spooling sends mails on the end of each request, so there is not much to monitor.

I created a small shell script with no dependencies other than basic Linux tools (like find and wc) which counts the number of mails in the queue and let you thereby monitor them with Nagios/Icinga. For my example I'll use check_nrpe to execute the plugin remote on the application server.

First, here is the script. You need to save it executeable in your plugin directory on the remote machine, which in my case is /usr/lib/nagios/plugins. I saved it as check_spool:

Next you need to configure the plugin on your remote machine, adding the command to the nrpe commands in your nrpe.cfg:

command[check_spool]=/usr/lib/nagios/plugins/check_spool "/path/to/swiftmailer/spool" -w 5 -c 50

Mind the /path/to/swiftmailer/spool which should be replaced by the directory you keep your spooled files in. The -w and -c options are the Nagios/Icinga standard parameters for warning and critical threshhold. With this command registered, you can restart your nrpe server and add the check to the monitoring server:

define service{
    use                             generic-service
    host_name                       hostname
    service_description             Swiftmailer Spooling Queue
    check_command                   check_nrpe_1arg!check_spool

You can place this anywhere in your icinga/objects folder. Change the hostname for your defined hostname and restart Icinga. Maybe you need to change the check_nrpe_1arg for check_nrpe, depending on your Nagios/Icinga version.

Now, looking in your Nagios/Icinga frontend, you should see the service running without issues. To troubleshoot, you can execute the script on the remote server manually and see if any errors are thrown. Doublecheck the command in your nrpe.cfg if parameters are correct. If NRPE is working, this command should be no exception.

Happy monitoring!

Donnerstag, 6. Dezember 2012

Advanced git: Different ways to exclude files

When I started using git, I very fast learned about the local .gitignore file. It's one of those handy features that everybody knows about and which is used in almost every project. In there you find excludes for build files, temporary files, caching files and so on.

Working with the local .gitignore can lead to a few problems though:
  • If you want to add specific files for each commiter (different IDEs or so) or OS (like the windows thumbs.db or Unix ~ backup files), your gitignore gets really messy 
  • If you add your IDE and then use another one later, you have to change gitignore files in all your projects
  • If you want to contribute to an OOS project, you are not supposed to change the gitignore, so you either not add those files to git and have clutter around or find another way.
And that's why there are different files which all can be used to ignore files within your git repository.

Sonntag, 25. November 2012

Doctrine MongoDB ODM: Storing arrays

When working with Doctrine MongoDB ODM, one thing I struggled to get right was arrays inside a document. I remember a few questions on StackOverflow who also didn't know how to handle this.

The answer is very trivial, but the documentation lacks a description, so I am posting it here (as well as adding it to the documentation of course):

class MyDocument
     * @Field(type="hash")
    private myArray = array();

There is a Mapping Type called "hash" which mappes a PHP array to a MongoDB one.

Of course you need to remember that an array in a MongoDB document cannot have any keys but only values. If you need keys, that would be a new document which gets embedded (see the docs on embedded documents).

Sonntag, 18. November 2012

Using 37signals/sub inside a vagrant virtual machine

Todays topic may be a bit specialized, but I guess it could be interesting for some! 37signals released their shell script toolset (which has a very small footprint) to the public calling it "sub". You can find the code in their github repository and the relating blog post on their signal vs. noise blog.

As I am running windows, it seems that this toolbox is useless for me. But I am working with vagrant all the time, so there is always a linux running in a virtualbox. It goes without saying that using sub within a vagrant box is not as comfy as running your shellscripts locally, but it's better than scattering your scripts all over the place.

When trying sub, I realized that as SharedFolders cannot contain symlinks (at least this is the case when running vagrant on a windows host), sub will not work. In this blog post I would like to describe a setup which makes it possible to run sub within your vagrant box.

Donnerstag, 8. November 2012

symfony2, Behat, Mink & Zombie.js: Headless browser tests

I wanted to work with Behat for a while now. I remember hearing a talk by @everzet at the Symfony live Paris this year which got me interested in Behaviour Driven Development (BDD) and Behat. Now I finally came around and looked into the whole topic.

I decided to go straight to browser tests which also work for javascript heavy sites as this is a real pain for me right now, using symfony and the TestClient it provides which of course cannot evaluate any javascript.

Sonntag, 4. November 2012

Contributing isn't just about code

As some may know, I am a big fan of Open Source. One year ago I always thought about contributing to some OSS project but never did it. Sometimes I would donate a few dollars for a project I really liked.

Then I started using github, and everything got way easier. I started using github for my projects, then had a look at others. I did a very exiting first pull request on a minor error I found in the symfony documentation. I did a few more since then.

In this article, I would like to point out a few ways on how to contribute to open source projects. Turns out there are much more ways to contribute than fixing a bug in the code or adding a feature.

Sonntag, 28. Oktober 2012

I will outsource (almost) everything!

Yesterday evening, playing around with the bitbucket issue tracker that comes with each repository, I realized what a fool I was for setting up a git server for our startup, PicturePlix. I realized that hosting your own mantis setup is just plain wrong. The same is true for the wiki and many apps more. Let me tell you why.

Samstag, 20. Oktober 2012

Living on the edge

Everyday development for a company usually involves using stable libraries which are in use for some time. Companies have (valid) requests to only use software which is reliable, without major bugs and well documented. So even when you can introduce a new library/tool, you'll most likely be able to work with it pretty flawless.

When you are on your own, coding away for fun or some weekend project, you can dive into bleeding edge technology, using alpha/beta software like crazy and get away with it. Why you ask? Because it's fun and you'll get a better developer.

Montag, 15. Oktober 2012

Build systems > Own scripts

For a long time I though build systems are overhead. You can always write a (insert language of your choice here) script and run it to test or build your software.

Setting up phing to build and test a symfony 2 project of mine really showed me the value: These systems (for example ant, a java build tool very well known) are designed to do exactly this one task, and they are designed to do it very good.

Sonntag, 7. Oktober 2012

symfony2: Testing secure pages

If you develop a web application, more often than not you have some kind of user section or admin panel where some kind of login identifies the user and protects your actions against usage from unauthorized people. It can be difficult to do functional tests with this kind of pages as you need to simulate some session or cookie context.

In this tutorial, I want to show you how to test your functional pages with symfony2 and phpunit. If you are experienced, this cookbook explanation might be sufficient: How to simulate HTTP Authentication in a Functional Test. For me it wasn't, first because I didn't found it and second because it is very short.

Dienstag, 18. September 2012

symfony 2.1, Commands and Swiftmailer memory spooling

Working with symfony 2.1 for a while now I can say I really enjoy it. Composer is awesome and the new profiler bar is also very nice. I don't use forms that much so I cannot say anything about the new form component.

One problem which took me some time till I figured it out regards the sending of emails within a command when memory spooling is configured for swiftmailer. It just doesn't sent any mails! The reason is very simple once you look in the right direction: Memory spooling means the emails get sent on the kernel terminate event, which never happens when a command is executed. The issue is discussed on github.

Donnerstag, 13. September 2012

symfony2: Testing email sending

On my endless struggle to achieving 100% code coverage and catching every possible (and impossible) test case (I'm just kiddinh, please don't do this!) I was in need for some way to test the sending of emails. This blog post will show you how to test this within a symfony2 (I'm using symfony 2.1 RC2) application, using the default swiftmailer.

Let's be clear hear: I'm not talking about unit tests. These have to be done in swiftmailer or the symfony swiftmailer bundler. I'm talking about functional tests of your application where you want to check if a particular action by your user (or your admin, or your cronjob, or whatever triggers the sending of an email) does lead to an email being sent.

Mittwoch, 5. September 2012

Ship it on day 1!

I don't know what's the matter with us web developers. We have the luxury of being able to release a product and improve it many times without any concernc. We are luckier than software developers who must ship their product through an update mechanism and we are much luckier than any other engineer, who simply cannot improve the product much when it's built.

Yet we stick to the old concept of developing and developing, not shipping till it's "really" ready. This "ready" means a shitload of features nobody will use, a ui most of the users cannot grasp and a long time till any user can experience your application.

I would bet serious money on the fact that most developers agree to this and yet won't do it. One thing which always comes up is the set of critical features. To me, there is not one feature which is critical. Just get up the first page and get it out there. Set up a signup page for beta users. Start implementing the first feature, create an account for yourself and start using it. If you think it's worth trying for somebody else, let them have access to it.

If you start implementing a new feature, get it out there and just let a selected group of users access it. See what happens. You can polish it later. You can add the missing subfeature later. Nobody will stop using your service just because there is some object which cannot be deleted right now. Or updated. Nobody will stop using it because there are some design flaws.

People will stop using your product when they wait 1 month for a set of features and then get crappy functionality they don't need.

On the other hand, people will love you when they feel they are in control: They mention some improvement and the next they it's there. It may not be perfect, but they can achieve their goal. They can use it and see what can be improved.

Or, as the guys at opbeat say: Fuck it, ship it!

Sonntag, 26. August 2012

Sleeping habits

I am one lucky guy who almost never has problems to sleep well. Often it's to short but I fall asleep very well and don't wake up often. But, because of the short time I sleep, I was tired often. And I realized that even when I sleep enough, I'm still tired in the evening.

Reading from many people that they get up very early I thought about giving it a try.

Donnerstag, 19. Juli 2012

New pecl mongo package breaks mongodb-odm

Using vagrant has many advantages. The fact that every boot updates your packages is not one of them. It  means you might end up with a new version of a package that breaks something even thought your whole setup worked great yesterday.

When booting my vm today, the pecl mongo package 1.2.11 got installed. And it breaks doctrine/mongodb. The error message reads as followed:

Function MongoCollection::setSlaveOkay() is deprecated

As for a solution, Jeremy suggest to set error reporting to not include deprecated messages. This might work for some. People developing symfony2 applications are having a harder time since the kernel sets error_reporting to -1 when in debug mode.

If you are facing this particular error, you should check if error_reporting is excluding deprecated:

error_reporting = E_ALL & ~E_DEPRECATED

To run your symfony2 applications again, you got two options (I came up so far):
  1. Remove the debug mode from dev and test environment
  2. Remove the error_reporting(-1); from the bootstrap.php.cache and you are good to go if you don't rerun bin/vendors install (or php composer.phar install, if you are using composer).
I'm updating this post if the doctrine bundle is working again. Wanna get this news before encountering them. Follow me on twitter!

Dienstag, 17. Juli 2012

Web Development with vagrant and chef

If you are like me, you got your development environment set up the straightforward way. Everything is running local (including any lampp/xampp/wampp stack), you may have a git repository with your source code and some IDE or prefered editor to edit your files. There are many problems related to this approach, one which you encounter as soon as your project gets a bit more complex.

I faced this problem when I started working for a small software company while studing. I should work on a Java application which served a webshop-like aftersales website for one of the biggest German automotive companies. I spend almost an entire week setting up my PC so I could run the project in my browser. I got it working, but there where some bugs which did not occur on the other developers PC. I lost another week trying to solve this. And, and this was real fun, parts of the software didn't behave like on the production servers. The main problem was that nobody could tell me the exact steps to set up an environment which worked. It was all like "Well, you need JBoss" and "I have PostgreSQL installed that way, maybe that's the problem". You can imagine my frustration from day 1.

So, if the above describes your setup, you may want to think about doing it another way.

Donnerstag, 21. Juni 2012

Try, delete, repeat

Yesterday, I thought it would be fun to play around with Apache Solr. There is a php library and a symfony2 bundle, so I got solr up and running in no time. A few tutorials later I had some data from on of our projects in solr and used it for our search engine. Today I deleted the branch and removed solr*. But I would do it again. And I highly recommend you to do the same.

Tortoisegit 1.7.10 crashes constantly

After upgrading to Tortoisegit 1.7.10 I got constant crashes of Tortoisegit. Today it started crashing my explorer.exe as well. I tried the as well as the beta of msysgit 1.7.11 without success. I uninstalled tortoisegit and installed it afterwards, which also didn't help.

The thing which brought stuff back to normal was to disable the icon overlay cache. I did this using Tortoisegit Settings => Icon Overlays => Status cache: None.

It seems my overlay icons are not updating anymore, which sucks. But it's better than not being able to use Tortoisegit at all.

I can not find any way to clear the whole cache and see if this helps. If you happen to know how to do this, please leave me a comment or email me. Thanks!


Today I recieved an email from one of the Tortoisegit Core Developers. We could not track the problem down as I have no crash reports, but he pointed out that git gc might help. And it did! Running it in the repository I had problems commiting to solved the issue.

I figured that cleanup is nice to do some time, so I looked for a script. Chris Irish has one at hand:

find . -type d -name .git | while read dir; do pushd "$dir"; git gc --prune; popd; done

And for those of you that run windows (works in cygwin):

find . -type d -name .git | while read dir; do pushd "$dir"; git.cmd gc --prune; popd; done

This iterates through all folders starting with the current one and executes git gc --prune if it's the root of a git repository.

If you can reproduce this issue, make sure to get a crash report and to fill a bug report at the tortoisegit issue tracker.

Dienstag, 12. Juni 2012

Welcome disqus

You might have noticed it. From now on comments on posts can be made through disqus. As disqus is used on many blogs and the login can be made using facebook, twitter, openid and many more it should be no problem for people who want to share their thoughts.

If you don't like disqus, you can always email me at sebastian (dot) goettschkes (at) googlemail (dot) com. I'm happy to discuss various topics through email, so feel free to drop me some lines.

And, of course, I hope to get more comments now that disqus makes it easier to enter the personal information!

Dienstag, 5. Juni 2012

symfony2 Test Database Best Pratice

When you build a reliable testsuite, you get to a point where it gets really slow. Unittests may be fast, but then the functional tests get executed and it takes time to set up clean databases and a clean environment before a test can be executed. With over 900 tests, one of our projects took way over 10 minutes to build on our Jenkins server. You probably know that XP teaches to have a 10 minute build. So what to do?

Mittwoch, 30. Mai 2012

msysgit 1.7.10 and github

Today, I wanted to fetch a github repository through https, leaving me with a "Couldn't resolve proxy '(null)' while accessing".

Googling a bit brought up this discussion and editing the C:\Users\MYUSSER\.gitconfig, removing the [http] lines did the trick.

Montag, 21. Mai 2012

Start working on a "Weekend Project", not matter what it is

I found a blog post on called The unexpected benefits of weekend projects and it's a really good read. Beside some technical aspects I don't want to dive into, the article states one should totally do weekend projects, and I agree. But there are some "rules" about weekend projects I don't think you should follow.

Montag, 30. April 2012

Extending the Testclass for Unittests

In one of my last articles on Testclasses for symfony2 I explained some of the classes I use for my tests. Since then I found a great article on metatesting and want to update my UnitTest class to show some practical examples.

Dienstag, 24. April 2012

PHP Garbage Collector and Debian/Ubuntu

I recently stummbled upon a error message in the apache error log that looked like this:

[...] PHP Notice: session_start(): ps_files_cleanup_dir: opendir(/var/lib/php5) failed: Permission denied (13) in [...]

As I wasn't aware what that meant, I researched a bit, finding a php bug issue as well as a ubuntu bug issue. In summary, while php tries to clean up old sessions, debian and ubuntu servers permit the www-data user to do this in the session folder resulting in the error message stated above. The funny thing is that debian/ubuntu have a cronjob which cleans up the sessions, so everything works fine and if you don't look into your apache error log, you don't even notice what's going on.

So, what I did was to set the session.gc_probability in the php.ini to 0 so the garbage collector never tries to do a cleanup. As both the php guys and the debian guys are not willing to discuss this issue and come up with a default way to do this (either lowering permissions on the /var/lib/php5 or using the cronjob as default), I think this is a good way. As said, this should have no impact on your applications.

Montag, 16. April 2012

Deleting not existing remote repositories from Tortoisegit

I struggled with the following problem for some time, so maybe this will help people with the same issue.

We use git for version control and Tortoisegit as gui on top of git. Each developer has many local branches. If something needs to be shared, we push the branch to the remote repository, making it a remote branch.
If somebody deletes a remote branch, it shows up in Tortoisegit for all other developers so everybody ends up with many remote repositories which actually do not exist anymore. Furthermore, trying to delete this entry results in an error message because the branch does not exist on remote (yeah, I know, this is why I want to remove it from my lists, silly!).

To get rid of these, do a fetch on the remote repository and check "prune". Tortoisegit will recieve the information which branches do not exist anymore and delete them from all lists. Your remote branch list is clean and up-to-date again :-)

Happy git-ing!

Donnerstag, 12. April 2012

Testclasses for symfony2

So, when developing with symfony2, I rely on my tests. They are my safety net and without them, I get a little nervous after every change. Does everything work? Did I forget anything? So I developed some classes which I extend. They work on top of PHPUnit and the symfony2 WebTestCase. The classes are used by my different types of Tests: UnitTests, ValidationTests, FunctionalTests (as well as IntegrationTests).

Freitag, 23. März 2012

Using vsfstream (with symfony2)

I read about vsfstream when skipping through the phpunit docs. Back then, I decided I don't need a virtual file system. Some time later, I had to test classes which read and write files and found myself creating and deleting temporary folders, messing around with nasty errors (like my favourite one where for some reasons tests fail when I don't use @runTestsInSeparateProcesses).

Dienstag, 6. März 2012

Clean Code #3: Beware optimization

This time I want to talk about optimization and what Clean Code Developers has to say about it. It relates to the KISS principle I talked about last time!

There are two rules which apply when thinking about optimization:
  1. Do not optimize
  2. No really, do not optimize

Mittwoch, 29. Februar 2012

Clean Code #2: Keep it simple, stupid

The last issue I brought up was DRY, and this time it's gonna be about KISS. In this series I talk about my journey to become a better developer. I point out lessons I learned from good and from bad software.

Keeping it simple

So, KISS is for "Keep it simple, stupid" and it makes itself pretty clear. Just stay simple. Don't get fancy when you don't have to. This idea comes from an agile perspective, where a big design in the beginning is replaced by incremental design while the development takes place.

Samstag, 25. Februar 2012

Symfony2 & MongoDb & Indices

I'm right now creating a application using symfony2 and MongoDb. I'm new to MongoDb and have a long history with relational databases, so I struggle with some details on how non-relational databases work.

One particular problem was that no matter how I defined indices in my documents, they wouldn't show up in my database and would not work at all. I didn't got any exception, so I was a little lost.

After an hour of research I found I have to create the database with the following command in order to get the indices work:
php app/console doctrine:mongodb:schema:create

The "problem" is that as Mongodb creates databases and collections on the fly you don't need to create them. On the other hand if you don't create them manually, the indices are not created with the collection.

I hope this information helps someone also struggling with it.

Montag, 20. Februar 2012

Clean Code #1: Don't repeat yourself

I started this series with my first post and some statements but without real techniques. This series is about applying clean code in the real world. I will use the clean code developer rules simply because i think they gather it up pretty well. The page is in german, but I'll explain the topics I'm talking about.

Freitag, 17. Februar 2012

The path to clean code

Good software is not software that works. Well, it should work of course, but there is much more to good software than simply that it behaves like expected. Those who read "Clean Code", who worked on an agile project or who studied XP know this already. And I want to add my points of view to this discussion.

Donnerstag, 19. Januar 2012

Binary vs. ASCII

Just a reminder for everybody out there trying to upload files in php via ftp: Make sure to double check if you upload binary or ascii. I wrote a small Class which handles up- and download pretty well and has ASCII as default, and guess what took me half an hour to figure out? You guessed right: The pictures I was uploading where just a mess on the ftp server.