Ruin The assorted ramblings of Brendan Tobolaski

Running your own servers

The common wisdom, which Justin suggests, is to go directly to a highly abstracted, proprietary cloud service or a higher-level hosted back-end — the kind that are so high in the clouds that they call themselves “solutions”. But the “BaaS” landscape is still very unstable with frequent acquisitions and shutdowns likely, and hosting on VPS-plus-proprietary-services clouds like Amazon Web Services or higher-level services like Heroku or App Engine can get prohibitively expensive very quickly.1 Developers who build everything on these services by default would probably be shocked at how cheaply and easily they could run on dedicated servers or unmanaged VPSes.

Running your own servers really isn’t hard. Most developers reject the idea outright without even trying because it’s unfamiliar and intimidating. It’s considered an extreme, horrible, unfathomable situation that must be avoided at all costs, usually by people who have never tried it.

But that’s a fallacy. There’s a learning curve and necessary integration work for every back-end option, from iCloud and Dropbox to your own collocated servers. AWS, Azure, Heroku, App Engine, Parse, and similar services aren’t free, easy, or automatic. (Neither is “scaling” with them, regardless of what you’ve heard.) Hosted infrastructure is like sync: it has a minimum, unavoidable level of complexity to accommodate. You can’t just check a box or set a BOOL and have it all taken care of for you.

— Marco Arment on marco.org

I very much agree. At work, we deploy to bare metal hardware (mostly). We have automated deployment systems that run on git push. It works great. It did take some effort to get up and running but, it works better for our needs than anything else would.

My own projects, like this site, run on Mac Mini running on Ubuntu and hosted by Mac Mini Vault. Before that, I ran a number of VPSes on varying providers like Digital Ocean(affiliate link), Linode and Rackspace. VPSes are great to learn on. If you do happen to mess up, you can always throw it away and start from scratch. Its pretty forgiving. The one issue I have with VPSes is that their performance feels pretty inconsistent.

It does take some time to get everything up and running. Once its up and running, you don’t have to do too much. You do need to keep the packages updated. You have to figure out how to do backups. Eventually, you have to deal with the logs building up or hard drives filling up. Its not set and forget but, I don’t think any platform is really set and forget.

Eventually, you’re going to run into issues with anything. When that happens, you’ll want as much access to your servers as possible. Having set things up yourself, you’ll have a pretty good idea on where to look to find the problem.

I do take issue with one thing that Marco says:

6. I also maintain a giant shell script that configures everything I need on a clean install of CentOS. Once you’re comfortable with the basics, I suggest doing this, as it makes it easier to set up new servers or switch hosts. Making such a script is much easier with VPSes, where you can start a new one, test it out, change it as needed, delete the instance, and try again on a clean one. Ideally, your servers should be disposable and easily recreated. The only backups you should need are your source code (which should include any required server-setup scripts) and your database’s data.

There is no reason to make this a shell script (The rest of this is great). There are a plethora of tools that are designed for this exact purpose: Ansible, Chef, Puppet, and Salt. I’ve used all of them except Salt. They are all good and all fit into their own niche.

  • Ansible is by far the quickest to get started.
  • Chef is best for extremely large deployments. It seems to be leading in Infrastructure as code space. It feels the most like programming.
  • Puppet seems to be about in between Ansible and Chef. It doesn’t feel quite as much like programming as Chef does but it does seem to be quite a bit harder to get going with than Ansible. I found their getting started guide to be horrendous. Puppet is what we use at my day job.

If you are looking to get started with one of these, I’d recommend getting vagrant set up. It supports all of these configuration management tools and it will allow you to rapidly develop and test your scripts.

Running Ghost as an npm module

Starting with the most recent release, 0.4.2, Ghost is now published as an npm module. I found the upgrade instructions to be rather horrifying and it just makes sense to abstract away Ghost core. There isn’t a reason to modify core, just like WordPress. There is a wiki page for running ghost as an npm module but, they don’t include everything that is needed.

I’m starting with the assumption that you have your Ghost site in git already.

  • Delete everything in you the sites folder except config.js and your content directory.
  • Add a package.json file:
{
  "name": "blog name",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "start": "node app.js"
  },
  "dependencies": {
    "ghost": "^0.4.2"
  }
}
  • Modify your config.js file to add to add a paths section like:
paths: {
  contentPath: path.join(__dirname, '/content/')
}

To both the development and production sections

  • Add an app.js file:
var ghost = require('ghost');
var path = require('path');
ghost({
  config: path.join(__dirname, 'config.js')
});
  • Install the dependencies by running npm install --production
  • Restart/Redeploy your blog and enjoy running Ghost as an npm module

Start Ghost using Upstart

I spent some time looking for a script to keep Ghost running. I had 3 things that I wanted to accomplish: to start Ghost on boot, restart Ghost if it dies and to not run Ghost as root. I found a whole bunch that fulfilled the first. A few that fulfilled the first two. The page on deploying Ghost has a few that fulfill the first two. There are way too many that expect you to run Ghost as root, which is not a good idea. Here is the upstart script that I came up with to fulfill all 3 requirements on my Ubuntu server.

description "Ghost Blog"

console output

start on runlevel [2345]
stop on shutdown

respawn
respawn limit 99 5
script
  cd /path/to/ghost
  sudo -u deploy /usr/bin/npm start --production 2>&1
end script

I’m sure, its not great. It probably could be improved. If you know something that could make it better, please leave a comment on the gist

The NSA plans to infect millions of computers

In some cases the NSA has masqueraded as a fake Facebook server, using the social media site as a launching pad to infect a target’s computer and exfiltrate files from a hard drive. In others, it has sent out spam emails laced with the malware, which can be tailored to covertly record audio from a computer’s microphone and take snapshots with its webcam. The hacking systems have also enabled the NSA to launch cyberattacks by corrupting and disrupting file downloads or denying access to websites.

The implants being deployed were once reserved for a few hundred hard-to-reach targets, whose communications could not be monitored through traditional wiretaps. But the documents analyzed by The Intercept show how the NSA has aggressively accelerated its hacking initiatives in the past decade by computerizing some processes previously handled by humans. The automated system – codenamed TURBINE – is designed to “allow the current implant network to scale to large size (millions of implants) by creating a system that does automated control implants by groups instead of individually.”

— Ryan Gallagher and Glenn Greenwald at The//Intercept
</figure> This is awful. It should not be allowed to happen. Spying is one thing but, deliberately infecting systems is illegal and immoral. The NSA should not be allowed to do this.

Own Your Words

The issue for me isn’t legal rights to the content, or revenue generation, or even control over advertising. Those are all legitimate concerns, but they’re also valid for platforms where other people host your own personal blog, and I don’t have a problem with those. Likewise, writing words that others pay for is a choice. No problem there either.

I do have an issue with giving away your words just because it’s easy to do so. By default, your words should be yours, and that means more than being attributed: it also means gathering them together in your own publication, controlled by you, that serves as a place for your own voice to be heard above (and instead of) all others.

Matt Gemmell

I very much agree. This is why I run my own blog, both here and on tobolaski.com. I feel like if you want to write, you should take credit for it. You also also own your online presence. You should own the domain. You should be able to choose how your words are presented.

I would never use something like Medium. While it looks great and great to use, it is made to be exactly opposite of what I just described. Building an audience is tough but, its a worthwhile pursuit.

Docker Appliances

I recently enabled comments using discourse. Continuing with my recent quest to Docker all the things, I did this using discourse_docker, which allows you to fire up a docker container with Discourse running in it. It works great.

docker all the things

I think that Docker goes to be the next big thing in server management and virtualization. In particular, appliances could be a big thing. In this instance I didn’t have to know anything about setting up Discourse to get it up and running. I just had to follow the short set of instructions and now its whole stack is up and running with optimizations done by the creators.

This sort of thing is extremely powerful. It allows you to run great software easily. I have absolutely no experience in setting up Postgresql and yet I’m running a container with Discourse in it.

Dianne Feinstein Complains about the CIA

Senator Dianne Feinstein is frequently exasperating. The Democratic senator from California is one day ultra-liberal, in the lead in calling for gun reform. The next she is ultra-conservative, one of the staunchest defenders of the embattled National Security Agency.

The senator's contradictory nature was on show for all to see on Tuesday, when she delivered an extraordinary speech from the Senate floor. It amounted to the biggest and most public rift between Congress and the spy community since the 9/11 attacks. Ms Feinstein, who chairs the Senate intelligence committee, which has oversight of America's myriad spy agencies, accused the CIA of breaking into the committee's computers. It is an extremely serious charge: a breach of the constitution, the executive branch tampering with the elected branch. She described it as "a defining moment for the oversight of our intelligence community".

The Guardian

As the biggest defender of the NSA programs, it seems pretty hypocritical to complain about the CIA infiltrating her computer systems. Certainly, what the CIA did was illegal but, so are the NSA’s programs. If she wants to complain about this, she should also be against the illegal NSA programs. I think this sums it up pretty well.

</a> — Source Glenn Greenwald on Twitter

None of that should downplay what is actually happening. The CIA is spying on the people that oversee them. Both the NSA and CIA have greatly exceeded their authorizations. Neither of these organizations should have the power that they do and now it appears that they are willing to fight to keep it. The CIA is willing to spy on even their biggest supporter. This should not be acceptable under any condition. Even if you think that the surveillance is something that is needed, this should not be allowed. The intelligence agencies need to have oversight. Dianne Feinstein is on the oversight committee and as such needs to be free from this sort of activity. The Intelligence agencies need to be reigned in.

Microsoft's Missed Opportunity

On the eve of the launch of Microsoft’s major exclusive launch, Titanfall, it has occurred to me what a major opportunity Microsoft has missed. This generation, both the PS4 and Xbox One are basically gaming pcs. Microsoft should have had a huge advantage as they power around 90% of desktops. Microsoft could have created a unified gaming platform.

The Xbox would have the reference design. Most people would have gotten it but, other people would have had the option to build their own. It certainly wouldn’t have been easy. There are a bunch of hurdles for them to jump through in order to get it to work.

There is the huge issue of drivers. This alone is probably enough to prevent this unification from ever happening. Then there is the performance differences. Console’s typically get much better gaming performance out of the same chips. They might have been able to just ignore this one if (this is a big if) the console’s performance could remain the same. A custom build would require much beefier hardware than the console. thats just the beginning but, that would have been a huge win for Microsoft.

I realize that would have been extremely difficult to pull off. Microsoft did have another option that would have worked nearly as well, release an Xbox Store on Windows. This would allow game publishers to allow you to purchase a single copy of the a game and play it on the system of your choice.

That would have been great for gamers. The first option would have been amazing but, it won’t happen. The second option, sadly, also won’t happen. It would be great for gamers but, publishers would never go for it.

How the French do cancer treatment

My parents were pleasantly surprised by his new routine. In New York, my father, my mother and I would go to Sloan Kettering every Tuesday around 9:30 a.m. and wind up spending the entire day. They’d take my dad’s blood and we’d wait for the results. The doctor always ran late. We never knew how long it would take before my dad’s name would be called, so we’d sit in the waiting room and, well, wait. Around 1 p.m. or 2 p.m. my dad would usually tell me and my mom to go get lunch. (He never seemed to be hungry.) But we were always afraid of having his name called while we were out. So we’d rush across the street, get takeout and come back to the waiting room.

So imagine my surprise when my parents reported from Paris that their chemo visits couldn’t be more different. A nurse would come to the house two days before my dad’s treatment day to take his blood. When my dad appeared at the hospital, they were ready for him. The room was a little worn and there was often someone else in the next bed but, most important, there was no waiting. Total time at the Paris hospital each week: 90 minutes.

‐ Anya Schiffrin on Reuters

Does anyone think our way is better? I would hope not. Cancer is awful in every way. Why is our method of treating have to be awful as well? Surely we can do better.

You would think that, given how much how well the patient is treated, the French treatment would cost considerably more but, judging from the county’s healthcare spending, they do not. We spend far more on healthcare than any other country and we don’t have the results to show for it.

“We spend one and half times more per person on health care than any other country, but we aren’t healthier for it.”
— President Obama

The situation is rediculous. There has been a huge amount of backlash against the Affordable Care Act (or Obamacare if you prefer). Most of the backlash seems to be that it goes too far. In all reality, it doesn’t go far enough. At the very least, we should have gotten the public option. Ideally we would have gotten single payer.

Kansas Gay Discrimination Bill

If that sounds overblown, consider the bill itself. When passed, the new law will allow any individual, group, or private business to refuse to serve gay couples if “it would be contrary to their sincerely held religious beliefs.” Private employers can continue to fire gay employees on account of their sexuality. Stores may deny gay couples goods and services because they are gay. Hotels can eject gay couples or deny them entry in the first place. Businesses that provide public accommodations—movie theaters, restaurants—can turn away gay couples at the door. And if a gay couple sues for discrimination, they won’t just lose; they’ll be forced to pay their opponent’s attorney’s fees. As I’ve noted before, anti-gay businesses might as well put out signs alerting gay people that their business isn’t welcome.
Mark Joseph Stern on Slate

In what universe is this right? This is terrible in every single way. If your religion tells you that you need to reject gay people, its wrong. Time to find a new religion, or better yet, realize that the sky fairies that you believe in, don’t exist.

I don’t understand the fear of gay people that clearly exists. Are you afraid that they might make you gay? A gay person can’t make you gay any more than an intelligent person can make you smart. What exactly have gay people done to cause all of this hate to be directed towards them? Surely the atrocities committed in the name of religion should make us more afraid of religious people instead of gay people.