How to run an IDE INSIDE a Vagrant VM

Vagrant is intended as a runtime environment for use with your development, not as the development environment (IDE, tools, etc) itself. But I thought it would be nice if I could run my IDE inside the VM. This would have several advantages, including: leverage IDE shortcuts for executing the app, make it easier to connect the debugger, and ease ramp up time for other developers.

So I looked into setting up a Vagrant VM with Netbeans. The necessary steps are downloading and installing Netbeans in a provisioner, and forwarding X11 from the VM to the Host.

There are many options to do silent installation of Netbeans. I had this inside my shell provisioner (note, this takes quite a while to download and install, and you should do this after installing java)

wget http://download.netbeans.org/netbeans/7.3/final/bundles/netbeans-7.3-javaee-linux.sh

chmod 755 netbeans-7.3-javaee-linux.sh

sudo ./netbeans-7.3-javaee-linux.sh --silent -J-Dnb-base.installation.location=/opt/netbeans-7.3

rm netbeans-7.3-javaee-linux.sh

To do X11 forwarding requires a few steps

• Add to Vagrantfile: config.ssh.forward_x11 = "false"
• run vagrant with “vagrant up --no-provision” so you don’t download netbeans again!
• vagrant ssh into the VM, running netbeans brings up the UI on your host display, it “just works!”

At this point, it works but performance is horrible. Clicking on a menu in Netbeans takes about 12 seconds to respond.

To improve the performance, we can specify a faster cipher and compression for X11. To specify this, connect with ssh directly instead of using vagrant ssh so we can pass in the ssh options. Use vagrant ssh-config to configure ssh to connect to the vagrant VM: vagrant ssh-config > ~/.ssh/config (be careful not to blow it away if that file already exists). The command looks like ssh -F ~/.ssh/config -c arcfour,blowfish-cbc -XC default  Note that the ssh config is set to “Host default” and we need to specify that (“default”) in the ssh command. Using localhost in the ssh command will not work. After this change, performance is much better but still bad (response time opening menus goes from 12 seconds to 4 seconds)

Additional things to try include manually modifying the VM video RAM to 64MB from 8, but that had no effect. (Indeed another VM I use for development that has the full gnome desktop installed and regularly run an IDE with no issue has only 12MB of video RAM) Another initially promising option was to enable VirtualBox 2D acceleration, but that is for Windows guests only so is not an option for Linux guests.

More potential steps you can take to improve the performance

• VMWare’s provider is rumored to have special graphics optimizations that let you work with an IDE inside a VM.
• You can try VRDP, it may have better performance than X11
• NX Server is supposed to be faster than X11

In conclusion, using Vagrant as your complete development environment does not really work out as well as you’d hope. Perhaps not so surprisingly, it’s best to use Vagrant as it was intended.

Clock Angle Problem Part II

Recently I posted a solution to the clock angle problem. At the end we were left with a few questions involving how to spice the problem up a little bit. In this post we will explore these question more. Namely, we will introduce a seconds hand along with the hour and minute hands, and we will explore caching.

First of all: what if the hour and minute hands ticked for every second and we wanted second-level accuracy? What we need to do is presume that there’s a second hand, and add the degrees changed for each second to the hour and minute components for each second tick. We can find the math for it, and write some code like this:

  // calculate all units in degrees per second

  // minute hand is 6 deg/minute plus 1/10 deg/sec

  double md = m*6 + s*0.1;

  // hour hand goes through 30 degrees per hour

  // plus 30/60=1/2 deg/min,

  // plus (1/2)/60=1/120 deg/sec

  double hd = h*30 + m*0.5 + s*(1d/120d);

  double result = Math.abs(hd-md);

  return result > 180 ? 360 - result : result;

Second of all, if the calculation was really expensive, how could we implement caching? (Hint: Not WeakHashMap) There are a couple ways to approach this.

It helps to understand the nature of what we’re computing. Is it time-intensive? Memory-intensive? Does it make sense to pre-compute values? In my caching example I took two approaches: one that pre-computes all possible values, and one that caches values as they are calculated.

It might make sense to pre-compute values if the set of possible values is limited and known, and the calculations do not adversely affect the startup time of the application. In this case, both are true. If the calculation took a lot of time or the amount of data to store was extraordinary (say, if we had a clock with nanosecond precision!) then this approach would obviously not be viable.

If pre-computing is not viable, we need a way to determine a way to store cached values. If we are already using a caching solution elsewhere in our application already, the choice is easy and we can just incorporate that. One options is Spring Cache, which  provides a nice abstraction over a couple implementations (see some handy posts here, and here). Additionally, you can include EHCache and use it directly.

However, if you are not already using caching and just need a quick one-off caching solution (on the way to a more mature solution as your needs grow... no need to re-invent the wheel), we can try extending LinkedHashMap as a simple cache. Check out how we can incorporate it directly into a caching layer over the ClockAngle class. Again, this is not a full featured cache solution, just a quick and dirty solution that may work for a focused part of your application for the time being.

Finally, if we have multiple classes doing the calculations and multiple caching techniques, it becomes apparent that there is duplicate code (in the core calculation, and in any input validation). It makes sense to refactor out common code and use the Decorator Pattern to compose these classes together. I think the final result is beautiful, simple, and easy to test!

Accessing MySQL Instance In Vagrant VM

I’ve been playing around with Vagrant to set up development environments.

One thing that I thought would be useful would be to connect to the database which is running inside your Vagrant machine from the host machine. That way it would be easier to inspect or tweak what the database was doing while the application is running with a GUI like MySQL Workbench. Additionally you could reset the database with a SQL script from the host without dropping into vagrant ssh.

It’s not hard, but there are a couple steps you have to do to make that connection work.

Set up port forwarding

Inside the Vagrantfile, the line would look something like this

config.vm.network :forwarded_port, guest: 3306, host: 3309

Make sure the MySQL user can connect from outside localhost

We can do this at provision time when you create the database and create the user. One way to do this is for the MySQL provisioner to reference a SQL file to set it up:

shell provisioner:

mysqladmin -u root password root

mysql --user=root --password=root --host=localhost --port=3306 < /vagrant/mysql_boostrap.sql

mysql_bootstrap.sql

create schema appdb;

create user 'dbuser'@'%' identified by 'dbuserpassword';

grant all on appdb.* to 'dbuser'@'%';

Rebind MySQL Host inside the VM

We need to rebind the mysql inside the VM. Edit your my.conf file (say, sudo emacs /etc/mysql/my.cnf) and comment out the following lines

# skip-external-locking

# bind-address

Connect

We’re ready to connect from the host! To connect from the mysql client on the command line, we need to use the TLS setting. Your command line would look like this:

mysql --user=dbuser --password=dbuserpassword --host=127.0.0.1 --protocol=TCP --port=3309

If you use MySQL query browser or MySQL Workbench, the connection form requires 127.0.0.1, not localhost in connection setting dialog

Voila! Now you can connect to MySQL from the client on your host to the server running inside your Vagrant VM!

Troubleshooting A Mysterious HTTP 500 Response

The other day I was modifying a Spring REST endpoint, changing the DTO creation method. My DTO in this case is a variable-depth copy of my domain object with hibernate proxies converted or stripped out. Strangely, when I accessed the endpoint to retrieve the modified DTO, I would get a 500 response but see no exceptions in the logs. What was going on?

Troubleshooting is all about isolation. First, replicate the problem (isolate the trigger). Then, find where the expected behavior changes from the actual behavior (isolate the location in the pathway). Finally, see what you can change in that location that resolves the problem (isolate the fix).

I’d already started isolating the trigger: creating a DTO with one technique vs another. I knew that I had changed how my DTO was being created and that the returned object was supposed to be the same as before. So to further isolate the trigger, my first question was: “What is the difference between the previously returned object and the new object?”

For comparison I created both objects in the controller method so I could compare the working and broken objects side by side. Stepping through the debugger with the broken code exited the Controller with the return of a valid DTO. But I was returning more information than before, specifically, three collections each of which had shallow copies of their elements. When I emptied the collections, the formerly-broken object was successfully returned without a 500. By process of elimination I determined with collection and which elements in it would trigger the 500.

Now that I’d isolated the problem to a new specific element in my response, the next step was to isolate the location in the pathway where the problem was occurring. From the work in the debugger, I knew that the problem was happening between the return of my controller and the receipt of the response by the browser. What happens between those two points?

Unfortunately I didn’t have the Spring source code readily available to step into, but I knew that the Jackson mapper was marshalling my @ResponseBody object to JSON behind the scenes. Maybe there was a problem with the marshalling?

I quickly created a unit test that instantiated a Jackson ObjectMapper and marshalled a DTO containing the problematic element that I knew would trigger the 500. And sure enough, there was a NullPointerException inside a “get” method being called by Jackson! It turns out that Jackson was picking up a “get” method and evaluating it during the marshalling, but the “get” method was actually performing a calculation instead of a simple property return. And the calculation involved an object property that had been set to null during my new DTO creation.

Some quick googling revealed that this is a known gotcha. The solution I zeroed in on was to rename my method so that it was prefixed with “calculate” instead of “get”, which was probably a more appropriate name anyway. And of course the unit test was perfect to show that this fixed the problem (isolate the solution).

So in turn I’d isolated: the trigger, the location in the code path where the problem occurred, and the change that would fix the problem. It was a pain in the butt, but a good exercise in troubleshooting.