Vanilla WordPress performance isn’t great, each page has to be generated via PHP per view, which soon starts to bog your server down under load. Fortunately there are some plugins and tweaks you can make to improve performance and resilience.
Any PHP or Apache gurus out there, feel free to comment and let me know where I’m going wrong, I don’t claim to be a PHP environment expert!
I’ve generally used W3 Total Cache which provides a number of different levels of caching, including the PHP itself if the relevant apache modules are installed. You can even delegate data to a CDN if that’s your thing.
But before installing the plugin, let’s get everything ready for optimising it.
To get the full benefit of a plugin like Total Cache, you need to add a few modules to Apache and PHP.
Install DotDeb repository
The DotDeb repository is set up for Debian boxes with up to date versions of common packages.
Instructions are straightforward and involve adding their repository.
Having been following the progress of a couple of Sheffield companies, OrderHarmony and Matchchat during the Ignite100 accelerator programme in Newcastle Upon Tyne, I was interested to find out what is planned for Sheffield.
This is my write up.
What is it?
Dotforge is going to follow the same kind of pattern as Ignite100, TechStars, Y-Combinator and other programmes. A 13 week intense programme consisting of a number of cycles, pitches, mentoring, training, building, creating, destroying, tweaking and learning. The programme will be for 8 startup teams.
By the end of the programme, teams will be able to pitch to investors in Sheffield & in London, to hopefully provide funding for their startup business to make the next big steps.
The programme will, in theory, work well for any businesses that are high-growth, scalable and (web) technology-enabled, regardless of market or industry sector. The programme’s focus is to identify startups that can get to the next stage of growth, and get them to it.
Having said that, the programme has the following as its main areas of interest.
Take advantage of the ‘Cloud’
Work as Business to Business (B2B) products
use Big Data or Social data
have the capability for high growth
have global ambition
consider Sheffield strengths, for example
Applications are encouraged from teams with more than 1 founder. There is no upper limit and typically there will be 3 to 4 people. Preferably people who already know each other or have worked together, but there are no real hard and fast rules.
A lone founder could apply and get on the programme, but usually people find that they are so busy during the 13 weeks, that the load needs to be shared out amongst other team members, you can’t be in two places at the same time.
A good mix of skills is required, having development skills as well as ideas is pretty much considered essential.
Each company on the programme gets £5k investment, per person (up to three team members) for a 6% stake in the company. The 6% is made up of the various investors who are funding the programme.
Around half of all companies on these programmes end up getting decent investment offers at the end, i.e. hundreds of thousands of pounds.
There will be an office in Sheffield city centre, where the teams will be located for the 13 weeks. The focus for the teams will be on the programme, trying to juggle other interests at the same time will always be possible, but you get out of the programme what you put in.
There are often benefits associated with being on an accelerator programme, credit for Amazon web services or Rackspace for example. This would enable growth without having to worry quite as much about infrastructure costs.
Applications are now open, the programme will commence after Easter 2013. The application platform is hosted by f6s and will be their standard process.
The selection process is in three stages, the online application review, telephone/skype interview, face-to-face interview, each of which reduces the number of applicants down to the final eight.
I’m certainly going to consider this very carefully to see whether I can form or join a team for this accelerator and whether it is the right time for me.
If you’re working on a startup, or have a cracking idea, then you really should have a look and consider applying!
There has been a fair bit of discussion in the press about where Sheffield’s High Speed station is to be situated for HS2. The leaks indicate that Meadowhall is the likely location. Most recently in the Daily Telegraph
I’d like to make a few points about why Meadowhall makes some sense for a new station, Tinsley Yard makes even more sense and then I’ll provide some other examples of getting the station in to the city centre instead.
High Speed 2 (HS2)
HS2 is a new High Speed line planned to be built over the next 20 odd years from London Euston, initially to Birmingham, before splitting in to two arms, one to Manchester, the other to Leeds (via the East Midlands and South Yorkshire). Speed is projected to be up to 225mph with an estimated journey time from the South Yorkshire station of 1 hour 10 minutes. The route to Leeds & South Yorkshire is due to be open by 2033
Current and near future plans for Sheffield
The fastest trains at the moment take 2 hours and 7 minutes to travel from Sheffield Midland station to London St. Pancras via Chesterfield, Derby and Leicester.
Following electrification of the Midland Mainline, projected times to London would be 1 hour 45 minutes within the next 5 years.
UPDATE - actually, electrification from Derby to Sheffield will take place by December 2021 in Control Period 6
Common Design Patterns for High Speed stations
Although currently the UK has very few High Speed stations (St. Pancras, Stratford, Ebbsfleet, Ashford), Europe, especially France has a lot more experience of building and siting them. The common design patterns which are generally used are:
Intermediate stations are on passing loops, allowing non-stop trains to pass straight through at maximum speed. This means the station is generally long and straight, requiring enough room for the loops too. Platforms for HS2 are planned to be about 415m.
Here is a train passing at high speed!
Stations are often out of town to allow easy access for a wide variety of people, including huge car parks for drivers. The equivalent for the UK at the moment is the ‘Parkway’ station, like at East Midlands.
Speed is rarely compromised, and when it is, it is usually because of important cross-connections and infrastructure, i.e. Lille Europe, where a massive new station was built with major disruption. So you rarely get through stations built on bends or in city centres!
A new UK example will be the station provided outside Birmingham, the plans are available for download
HS2 stations are going to be designed for speedy embarkation, the trains are nearly half a kilometre long, so passengers will be directed to the right carriage before the train arrives. There will be no last minute jumping on with these trains.
Stations shouldn’t be too close together. A purist route would have a terminus at Leeds, a station outside Birmingham and a terminus at London Euston, that would be the most efficent and that’s your lot!
Why Meadowhall/Tinsley Yard makes sense
Space - there is far more brownfield space around Meadowhall for building new infrastructure. There is also room to move businesses around if required.
Existing infrastructure - the station would be on the tram network, serving Sheffield and by then Rotheram too. The M1 travels right by on the Tinsley Viaduct.
It meets the common design patterns outlined above.
Area - it would serve South Yorkshire, not just Sheffield City Centre. Most of the official documentation I have seen references South Yorkshire, not Sheffield
If built in the old Tinsley Yard, there is a lots of good space for it to go. Access could be provided at either end.
Why it doesn’t make sense
Meadowhall is already busy as a transport hub, especially the M1.
It will add a good 15-20 minutes on to the journey time from the City Centre, giving a similar time to the new ‘classic’ electrified service anyway.
City Centre businesses need a city centre station for growth (so they say, I’ve not read up on any research in to this).
At the time of writing we do not know which route HS2 is set to take through South Yorkshire. Consultation is due in 2014.
High Speed routes though, again, have common design patterns
no sharp curves which would reduce speed unless absolutely necessary. These trains are not built to tilt.
gradients aren’t usually too much of a problem, compared to normal trains and routes. High Speed lines often have ‘motorway’ style gradients.
High density population areas to be avoided otherwise extensive soundproofing & demolishment is required.
Consequently, I’d be surprised if HS2 was to be routed parallel to the existing mainline route via Dore and the Sheaf valley. You probably could squeeze the infrastructure in and you might have an ideal alignment for the existing Midland station.
However, what is more likely would be a route around the East Side of Sheffield where the trains from the South originally came from, before the route along the Sheaf valley was built towards Dore. This is the route you might have experienced when engineering works force a diversion. You leave Sheffield to the North, turn right through Darnall, Woodhouse, before bearing South back round to Chesterfield past Staveley and Barrow Hill.
So, I could see a route which heads round the East of Sheffield, avoiding the centre completely, before heading North to Meadowhall to a South Yorkshire Parkway station.
City Centre station (Midland)
Let’s assume that the route can be built along the Sheaf valley, parallel to the existing mainline. What do you do when you get to Sheffield Midland station?
You can’t build underneath it, because that is where the Megatron is. Midland station is built on top of the river.
You could build on top of it but building a bigger station on top of a smaller station would make a bit of a mess.
You could completely knock down Midland station and rebuild it.
You could incorporate it in to the existing station, but some platforms would require demolishing and there would be extreme capacity issues for existing services.
If the route of HS2 heads around the East of Sheffield, then why not use the old Victoria station? The alignment is correct for incoming trains and you could have a dedicated station there with, say a tramlink to Midland?
Well, I think the problem there is that trains heading for Leeds would then end up pointing towards Hillsborough, Oughtibridge and Stocksbridge. You would need some serious infrastructure (i.e. a big tunnel) to get the route back to a more Northerly direction to continue the journey to Leeds. Not impossible, but adds a lot of extra cost.
Hybrid using Victora & Midland Stations
One off the wall idea, might be to re-open Victoria as a mainline station, re-open the line to Penistone, then divert some of the trains currently using Midland to a new Victoria Station.
Services to Lincoln, Leeds (Northern), Barnsley, Huddersfield could all use Victoria with Cross Country and East Midlands then using a reduced size Midland, creating extra space to build High Speed platforms there.
Of course, re-opening Woodhead would be the icing on the cake!
This seems to be the best location for alignment and existing space. The long lost Marshalling Yard is in a great location for the Motorway, not that far from Meadowhall and would probably be good for noise and local housing. I think there is a good chance that it will be used as the final location.
Someone has done a great route example here - looks very plausible to me.
The more I look at maps and consider routes, the more I think that a route passing across the valley parallel to the M1 is likely, rather than one parallel to the current mainline at Meadowhall. So, imagine a station under the Tinsley Viaduct, at 90 degree angles to the existing main line. Probably then, with tunnels either side.
If we end up with a station in South Yorkshire, then my vote goes to Tinsley Yard. If you want to get quickly to London in 2032 from Sheffield City Centre. Take the Electrified Midland Mainline to St. Pancras in 1 hour 45 minutes. It’ll probably be cheaper than HS2, be more scenic and will not take much longer.
If you don’t live in the City Centre and you drive, park up at the HS2 station next to the M1, get there early and whizz down HS2 to Euston in 1 hour 10-15 minutes. Either way will get you in to the centre of London fairly sharpish. Win win!
I was thinking about a Debian install on my server, but in the end, I thought I’d go with Ubuntu server.
Ubuntu server install
I’m not going in to all the details about installing Ubuntu server (debian would be similar), but I thought it would be worth noting the steps I made to get the RAID device partitioned ready for installing xen or KVM.
When booting the server, you are able to access the MegaRAID configuration. As my server came with 2 unregistered drives, it was simple to create a new RAID1 configuration with those two 146Gb drives, creating a virtual 146Gb RAID1 drive.
I ummed and ahhed a bit about whether to use the RAID controller, but it looks like it might be ok kit.
I usually use inetbootin to create a USB memory stick with the OS I want to install. In this case, I downloaded the 64bit Ubuntu image and used it to create the bootable USB stick.
Expert install followed.
Grub2 gotcha 1
Oh dear, I had a number of failed installs until I asked an expert (@wilbatron) for some help. The grub2 installation in to the MBR kept failing because… The grub2 installation doesn’t fit in the MBR does it? I wish it told me that rather than just failing.
Ah well, got it sorted in the end, here is how I set up my partitions.
Partition set up
During the install, do manual partitioning. We’ll set up 2 physical partitions, one of which will be a Logical Volume Manager (LVM) Physical Volume (PV)
At this point, the menu will show your USB stick (still mounted) and your 146Gb virtual RAID drive. The Hardware RAID is hiding the fact that you’ve actually got two hard drives there and the installer just sees one.
Select the Raid array - in my case, it was labelled 146.0Gb LSI MegaRAID 8708EM2
Once selected, create a new empty partition (and table) for this device
Choose GPT as the partition table type (not MSDOS!)
Now select FREE SPACE 146.0GB and create new partition
Set new partition size to be just 10Mb, this is where the installer will put the rest of grub2
Set to be at the beginning of the drive
Set type to be used as Reserved BIOS Boot area
Create next partition to use as Physical Volume (PV) for LVM
Now we configure the LVM
Configure LVM (option at the top of the partitioning page)
When prompted, write partition changes to disk
Create volume group, give it a name and select the PV.
Create logical volume, call it ROOT or whatever, mine was 15Gb - this is where the OS will be installed. You could create all the separate partitions if you like of course.
Create swap volume, I chose 6Gb, although the server has 12Gb of server memory, I didn’t want to give it swap of that size. We’ll see how we go.
Finish up and head back to partition page
Set root partition to be root (ext4), doesn’t need to be bootable partition.
Set swap to be swap
Down to the bottom of the page and write changes to disk
Back to main menu and install!
Installing grub in MBR - gotcha 2
Just another slight gotcha, if your USB stick is mounted as /dev/sda & your RAID array as /dev/sdb you might have to tell it not to install to the MBA, but instead to where you want it installed, i.e. /dev/sdb
BMC (Baseboard Managment Controller)set up
To get the BMC up and running properly, I needed to do a few things, first, is to ensure that it is set up correctly in the BIOS.
Enter the BIOS and in the server management section, ensure it is set to use the correct LAN port, either Dedicated if you are using the single port on the back of the server, On Board for one of the 4 built in LAN ports. I used the latter with DHCP. Make a note of the IP address which is given to the BMC and fire it up in your web browser. All being well, you should see the log in page:
Default authentication log in
Apparently the default is either admin/admin or ADMIN/ADMIN so give that a go first of all to see if you get in. If not, we will have to do some jiggery pokery on the server.
Resetting BMC credentials from the server
The tool, ipmitool is required to access the BMC from the server. I just did apt-get install ipmitool to get it up and running. However, before I got it working properly, I had to load some kernel modules
Kernel modules for ipmitool
First, I’d see whether it is working straight after install:
if you get
Could not open device at / dev/ipmi0 or / or dev/ipmi/0 / dev/ipmidev/0: No such file or directory
Get Device ID command failed
Unable to open SDR for reading
You can add these to /etc/modules for permanent loading.
Now try again, you should get some results
Get correct channel
ipmitool -I open channel info 1
(where the latter 1 is the channel, so if 1 doesn’t work, cycle through)
Channel 0x1 info:
Channel Medium Type : 802.3 LAN
Channel Protocol Type : IPMB-1.0
Session Support : multi-session
Active Session Count : 0
Protocol Vendor ID : 7154
which showed that Channel 1 was the one I needed.
ipmitool user list 1
should give you a list of users, then you can reset the passwords. I ended up setting all I could to password then, once I was in the web admin, I deleted the users I didn’t need and set all the passwords to be more secure ones!
Hmm, not sure I got this right, but I tried
ipmitool -I open lan set 1 password admin password
Password set for user 1
(the 1 here is the channel)
ipmitool user set password 2 password
for the various users until I could log in to the console!
I’ve been having a bit of a look around to try and find out where you can buy caddies and Seagate drives for the server.
Hard Drive Caddies
My server came with two drives in caddies, but with no spares.
According to the SuperMicro spec for the server, the caddy model number is MCP-220-00047-0B “Black gen 3 hot swap 2.5” HDD tray”
There seem to be a few UK suppliers around, although I wonder what has happened to all the drives removed from the servers before sale?
Anyway, here is one which I think I will try, from Lambda-tek, price currently £9.20 + VAT.
Then for the Seagate drives, mine has a couple of Seagate Savvio 10K.3 10k rpm, SAS drives, there are a few on ebay, this one seems to be the cheapest for used drives on ebay. Of course, these drives may have gazillion hours of use on them, but hey ho.
Well, I’ve ordered a couple of caddies and drives, will see how they turn out. Plus, I’ve got an Operating System installed, see next post!
Apparently, one of the three letter codes on the server can tell you where the server was located. Mine, seems to have SDC on it, but I haven’t been able to find out if that is a location or not.
This spreadsheet is the UPS delivery cheatsheet, no SDC on there.
Mine also has PDC on one of it’s stickers, which might be the Primary Data Centre. Try searching for it in your favourite search engine with Locog.
State of servers on delivery
The RAID card was loose in the server case when I unpacked mine, which I’m glad I spotted before firing up for the first time. Once re-attached, all seems fine.
I also thought mine was broken, but actually, it was what you might call a user error! What I hadn’t realised was that if you only plug in one of the power supplies, then the server lets you know that one of the power supplies might be broken by sounding a continous BEEEEEEEP! So don’t make that error, plug both power leads in!
Once again, @interpipes to the rescue! “beeping is just to tell you one of the supplies is “failed” - pull it out (an inch)/plug it in, and it’ll stop beeping.”
@interpipes also had some interesting configurations too on his servers
“Half of mine had bios passwords set, half didn’t, and 3 had a configuration on the BMC controllers.”
Advice on getting in to the BMC via @interpipes
The BMC is the Baseboard Management Controller, an extra microcontroller which servers to manage server kind of things at a lower level.
“Getting back into the BMC (which among other things gives you full IP KVM access),
first, set up the network configuration for it in the BIOS as if it has been configured it’ll have a static IP set the network config is under Server Management, and then the first option in that menu - I forget what it is called - I recommend using the dedicated LAN port setting, which is the solitary lan port above the two usb sockets.
Now you can get to it over your lan (pointing a web browser at it being the easiest way) but you’ll need to use ipmitool to change the root user password back to something you know (then you can log in and reset the BMC to factory defaults). Using the BMC it’s possible to mount an ISO from a windows file share (I think you can also mount one from a local disk when connected to the remote control interface) to install the system from entirely remotely.”
Installing an OS
Now that I could start mine up without it beeping the place down, I entered the RAID configuration and set up a simple 146Gb RAID1 set up, using the 2x146Gb 10k RPM 2.5” disks it came with.
I also used Unetbootin to get a server ISO set up on my old USB memory stick. Of course, it’s up to you to choose your favourite Linux distro, Unetbootin will allow you to download a customised ISO too which it will then set up on the USB, so you are not limited to the ones it comes with by default.
The server quite happily booted off the USB stick, so no optical drive is required (my server didn’t have one).
For the Olympics, a whole lot of techie kit was required to do lots of clever things. As a lot of the kit wasn’t required after the games were over, they have been made available for purchase at super cheap prices! Kit was originally for Educational Establishments, but once that batch were sold, I could buy one. So I did!
Server Model - Gateway GR380 F1
Well, I hadn’t realised that the Gateway brand had been brought back to life, but it has by Acer! The Gateway GR380 F1 - you can have a look at the spec if you like. It is a 2U (i.e. bigger) version of one of these coming with 2x146Gb drives, lots of memory and a big cardboard box.
What are you going to do with it?
Good question, I’m thinking of setting up a virtualisation kind of thing, so my big server can pretend to be lots of smaller servers, useful for building client sites, experimental things, build servers etc. We’ll see
Where do you start?
Another good question. It comes with the Smart Manager application, which you can install on it to do lots of clever things, however, at the moment, I’ve not even fired it up to see if it has the remnants of an Operating System on it. I’m suspecting not!
I’ve got one too, where do I start?
Well, I’ve managed to find the user manual, you can download it yourself from the Acer site.
If nothing else I’d like to have a look inside to see what sort of state it’s in. Then I need to work out how to attach the funky rack mounts to it, then I need to work out how to install Debian or Ubuntu on it (plus Xen probably for virtualisation).
Following on from getting Cathy & Heathcliff, our internet enabled thermal printers up and running as part of the Happenstance Project, I wanted to see if I could get them running on steam power.
Of course most electricity we get from the national grid has been generated by a steam turbine in a power station. I though, was thinking more of this kind of steam engine!
This is stationary steam engine, hand built in the West Midlands by a company called Mamod founded in 1937.
Mamod 1335D SP5
This specific model comes with a dynamo, dual cylinders and a generator, a 1335D. I wanted to see how much power it would output with the standard dynamo. The bulb was a 1.5V bulb, so I wasn’t expecting it to power an Arduino out of the box. I realise as well that the output would require stabilising and regulating. Cathy & Heathcliff’s Arduino’s require 7-16V.
And I was right! I lit the boiler using three of the solid fuel pellets and waited for pressure to build in the boiler.
Once the pressure did start to build, it soon became obvious that there were a few leaks which meant that some pressure was going to be lost. I kept on attempting to start the engine, but it was a while before it was ready, around 10 pounds per square inch of pressure.
After starting, the generator produced a rather paltry, 0.7V, hardly enough to light the light!
I managed then to stop the engine and hold it, to let the pressure build. Once it reached about 15 psi, I started the engine again and tested the voltage. This time it made it up to 1.5V
After that run, the pressure dropped and the fuel began to run out, so that was that.
The plan for next time will be to not even start the engine until the pressure reaches 15 psi. I might also try and extra fuel pellet to see if that helps maintain the pressure.
I’ve ordered some sugru to try and fix the leak too! Will be interesting to see if it works.
This year, Amazon launched it’s Pick up lockers and Collection locations. A great idea for a mail order company, providing delivery alternatives for customers so they don’t have to put up with the, generally awful, parcel delivery services. Having had parcels left outside houses and in rubbish bins, it’s good to know you can collect, at your convenience from a location you have chosen.
The Oxford - Cambridge Railway
Then, the other weekend, I was on a railway line I’d never been on before, the Bletchley to Bedford line, which joins the West Coast Main Line, to the Midland Main Line, serving lots of small stations on route. A lot of work has been done on the infrastructure recently, as services get extended from Bletchley towards Oxford, and hopefully one day, back towards Cambridge. The line will also be electrified as part of the ‘electric spine’.
About half way along this route, is a massive Amazon warehouse. Although it is next to the railway line, it isn’t rail served, there are no sidings set up for transferring goods. It got me thinking though, if local and major railway stations had Amazon pick up lockers, then they could be rail served in a really efficient way. Of course there would be infrastructure costs, getting Amazon warehouses linked up, but once that was done, a really efficient way of delivering goods.
Thanks to the Off The Rails hackday, I was able to team up with some fellow software engineers to build something in a day. Our team consisted of Frankie Roberto, Jez Nicholson, and Joe Hughes. Frankie had had the idea for building an application which would show sights of interest visible from the train route, indicating rough placements and which side of the train to look on. I knew Frankie from Sheffield, Frankie had worked with Jez before on a previous hackday and Joe was interested in what we were doing, our team was formed!
Interestingly, during the ‘speed dating’ section at the beginning of the day, when we chatted to other people about their ideas and what they were thinking of building, one chap, whose name I think was Phil, was thinking about Railway iSpy, which sounds pretty similar to what we actually built (without Big Chief i-Spy though!). Our application also bears an uncanny similiarity to the Peppa Pig episode The Train Ride!
After some discussion on ideas, scope, implementation and the Railway Performance Society we started building a Rails web application and an iOS app. Although I know a lot about railways, I’m still in the early days of learning Rails and iOS, so the patience of Jez and Frankie whilst building Scenic Railways was much appreciated. So whilst the three of us built the web application, Joe got cracking with building an iOS application.
We got an initial version built in the time we had (from about 11am until 7pm) before the presentations were due to start. I worked on a bit of map integration using leaflet.js whilst Jez and Frankie implemented the main structure.
Although the version we finished was relatively basic, there are some nice features, the best one being that the sides to look out of the window on, reverse depending on direction of travel! It could easily be extended in the future to add extra routes and extra points of interest (POIs). Joe had some ideas on making the iOS app competitive, by providing incentives for people to take photos of the points from the train!
So, feel free to have a look at the site Scenic Railways and you can even checkout the code from github and laugh at my Rails code (or lack of it!).