Friday, April 21, 2023

So you've finally read an EULA or ToS...

Important note to start with: I am not a lawyer, nor do I play one on TV, but I am a Web Developer, and have developed websites and applications that rely on boilerplate-style Terms of Service agreements like this one, so I have some experience in interpreting what's necessary for a site to run. My words and interpretation do not constitute legal advice and should not be relied on in a court of law.

Congratulations, you're now a little more informed, though given the dense legalese these contracts have probably not as much as you thought, and I'm fairly sure you're now freaking out about some clause that grants some kind of nebulously defined rights and licenses. So let's break down the licensing clauses in most common boilerplate ToS statements, and see what they say and why they say it.

By submitting, posting or displaying Content on or through the Services, you grant us a worldwide, non-exclusive, royalty-free license (with the right to sublicense) to use, copy, reproduce, process, adapt, modify, publish, transmit, display and distribute such Content in any and all media or distribution methods now known or later developed (for clarity, these rights include, for example, curating, transforming, and translating). This license authorizes us to make your Content available to the rest of the world and to let others do the same. You agree that this license includes the right for us to provide, promote, and improve the Services and to make Content submitted to or through the Services available to other companies, organizations or individuals for the syndication, broadcast, distribution, promotion or publication of such Content on other media and services, subject to our terms and conditions for such Content use. Such additional uses by us, or other companies, organizations or individuals, is made with no compensation paid to you with respect to the Content that you submit, post, transmit or otherwise make available through the Services as the use of the Services by you is hereby agreed as being sufficient compensation for the Content and grant of rights herein.

You represent and warrant that you have, or have obtained, all rights, licenses, consents, permissions, power and/or authority necessary to grant the rights granted herein for any Content that you submit, post or display on or through the Services. You agree that such Content will not contain material subject to copyright or other proprietary rights, unless you have necessary permission or are otherwise legally entitled to post the material and to grant us the license described above.

Woof, that's some legal jargon. So let's break it down:

By submitting, posting or displaying Content on or through the Services, you grant us a worldwide, non-exclusive, royalty-free license...

When you post something, you automatically grant them a non-exclusive (so it doesn't restrict what you can do with it, they have no rights to stop you posting it elsewhere or whatever) and royalty-free (that is, they don't have to pay you to use it) license.

...(with the right to sublicense)...

This is a tricky one, but it's there for a reason. We'll get back to it in a minute.

...to use, copy, reproduce, process, adapt, modify, publish, transmit, display and distribute such Content...

These are all ways of dealing with content that are necessary for a website or app to function. For instance, "reproduce", "copy", "transmit", "display" and "distribute" are all methods to get your content to other people. "Adapt" and "modify" are in there because your content might be reformatted (pictures cropped, for instance, or a PNG upload converted to JPG for better compression, or a particularly long message shortened with a link to view the full text) in some way to make it easier to display to other people.

...in any and all media or distribution methods now known or later developed (for clarity, these rights include, for example, curating, transforming, and translating)...

"Curating" would be making a "best-of" compilation, for instance. Transforming and translating would also refer to making content available in different languages than it was originally sent. And "any and all media now known or later developed" - If the Web 3.0 comes around, they still want to be able to send your messages to people in that too.

Now, back to that sublicensing part - When someone quotes your message to reply to it, they need a license to be able to reproduce what you posted. This allows them to grant that license on your behalf.

This license authorizes us to make your Content available to the rest of the world and to let others do the same.

Well, that's relatively simple. 

You agree that this license includes the right for us to provide, promote, and improve the Services and to make Content submitted to or through the Services available to other companies, organizations or individuals for the syndication, broadcast, distribution, promotion or publication of such Content on other media and services, subject to our terms and conditions for such Content use.

You grant them the right for them to make changes and make things better, to make ads for the service, and because the site enables embedding (that is having content from users shown on other sites, or shown as part of a news story, or whatever), the company needs the ability to grant those sites a license to display your content, same way you grant them a license to do so. This granted license is also governed by these terms of service.

Such additional uses by us, or other companies, organizations or individuals, is made with no compensation paid to you with respect to the Content that you submit, post, transmit or otherwise make available through the Services as the use of the Services by you is hereby agreed as being sufficient compensation for the Content and grant of rights herein.

If someone else uses your post, you aren't getting paid for it (at least, through here).

 And finally:

You represent and warrant that you have, or have obtained, all rights, licenses, consents, permissions, power and/or authority necessary to grant the rights granted herein for any Content that you submit, post or display on or through the Services. You agree that such Content will not contain material subject to copyright or other proprietary rights, unless you have necessary permission or are otherwise legally entitled to post the material and to grant us the license described above.

If you upload something, you are stating you have the legal right to do so, that you own it and/or can grant the licenses mentioned above. So uploading a movie would break this ToS (for example) unless you were the movie studio that made it. 

This is standard, boilerplate stuff you will find in every single Terms of Service agreement for any website or app that deals with User-generated content.

Thursday, December 15, 2022

Let me show you my ARSE.

Arbitrarily Routing Sound Engine

 I, and I'm sure a lot of other people, have a need that's seriously not being met by current systems on the market. I am referring to Windows-based systems here, as I'm sure you could probably do something like this with JACK on Linux, but ain't nobody got time for that.

What I need is a system that will allow me to map any number of audio inputs to any number of audio outputs simultaneously on Windows. So, for example:

  • Microphone, Discord audio, Game audio, Music and Overlay Audio routed to one input of OBS
  • Microphone, Discord, Game and Overlay Audio (but no Music) routed to a second input of OBS, for VOD capture
  • Microphone routed to Discord for chat (possibly game audio routed there too, so participants can "watch" without double audio confusedness/weirdness on their part)
  • Microphone, Discord, Game and Music all being captured simultaneously, in separate tracks, for later combining manually as part of an edited clip or highlight reel for YouTube
  • And, of course, Discord, Game, Music and Overlay all being played through headphones for the user to hear

This would also work well for Podcasters, being able to capture multiple microphones, and/or call-in guests, along with Soundboard audio in a multi-track session, without having to worry about sync or external sources, for later mixing and combining as necessary. You could also have plugins like an auto-ducking plugin, so when someone talks, another source (or sources) have their volume cut, like DJs on the Radio.

Existing Solutions

Currently it seems the best option is the Virtual Mixer software Voicemeeter (and it's increasingly more complex variations, Banana and Potato). While they can work, they're very limiting - There's no multi-channel recording, and they rely on having several virtual devices that you manually send audio to or consume audio from. While they do have recording built-in for Banana and Potato, it mixes the inputs down to a single output, which is not ideal. Software like this seems to be based on the same paradigm as hardware mixers - Mixing inputs down onto a (number of) output buses. While this works for some needs, it's not as flexible as it could be, and with a little work I think this could be a lot better.

Presentation (User)

So what does the user see? This is where it could get difficult. One of the easiest visual metaphors would be a patch board. Each (system) output would have a "default" option, so new programs/devices would automatically be hooked up to them, but you could also manually grab a "cable" and route an input to an output, or unplug a routed option if you don't want it.

To simplify presentation, by default, things that make sound would be stacked up on the left, devices that consume sound would be stacked on the right, and any plugins could be dragged into the space in the middle to sit between them. "Cables" then run between the sound makers and sound consumers, or from makers to plugins, and plugins to consumers.

This is still going to be a complex system, so definitely not for beginner users, but it would be immensely powerful, and would allow a great deal of flexibility, both for live presentation and for mixing and correction afterwards. For instance, the multi-channel capture would allow for easy editing, removal or replacement of content flagged by ContentID, for instance, having a good clean capture to re-assemble the audio from. For easier control, especially while other things are happening on the system (in a game, for instance) you could have some method to hook up and listen for MIDI control codes, so you can map inputs or outputs to things like the KORG nanoKONTROL2

Presentation (Programmatic)

So, how would this appear to programs running? Well, ideally there would be one input and one output (unless more were required, like in OBS' case). Each program would just see a single main and default device to output to, and a single default device to input from, and the engine would handle routing sources appropriately. The "devices" would accept the best quality option available (multi-channel, bitrate, and frequency) and internally the audio data would be converted as needed (Mono microphones upmixed to stereo, 7.1 audio downmixed to stereo for headphones etc.)

Potential Problems

The main hurdle here would be getting the audio out of Windows before it mixes it internally, as this system would have to bypass most of the internals of the Windows sound system, and presenting a unique audio feed to each listening program. I am unfamiliar with the internal workings of Windows Audio, but if programs like OBS can capture audio from a specific program it seems to me there must be a method to do so, even if it's making a "dummy" device that throws away any audio data it gets from Windows and hooking into programs themselves.

Conclusions

This would be a fantastic idea, and I'd love to see it implemented, but I cannot do it personally - It's coding on a much lower level than I've done before, involves hooking into the deep and arcane wizardry of the Windows audio subsystem which I haven't even looked at let alone touched, and is frankly way beyond me. But it's a good idea IMHO, and I'm sure someone could make a small fortune making it or something like it. I frankly wouldn't be surprised if someone has already made something like this, and if they have I'd love to know about it.

Meanwhile, feel free to use this idea to make the next killer audio app, and when you do just chuck my name in the credits somewhere. :D

Friday, September 24, 2021

The EU and device plug mandates

 Okay, so I'm hearing a lot of people grumbling about the EU's proposed new ruling to make devices (like phones, tablets and the like) all use USB-C. There's a lot of grumbling that it's "bad for innovation" (Thanks Apple for that one), and that maybe the UK had the right idea with Brexit, getting out from under these onerous regulations.

See, here's the thing, the EU started this crusade back in 2009. Then they had a "Memorandum of Understanding" signed by the main players in the mobile market (including Apple) that all the manufacturers would agree on a single standard for their devices, and stick to it. That way you wouldn't have to throw away chargers and cables every time you bought a new device, and as mobile devices were seen as more "disposable" than other devices (the whole "carrying around, getting used and worn every day" was what they were going with, more than "new model with new features coming out every so often") this would both reduce waste, mean consumers didn't have to buy a new charger and cable every time (even if it's included in the box, you're still paying for it), and promote innovation as third parties would only have to build for a single connector and it'd work with everything.

Back in 2009, the mobile space was vastly different, with a whole swath of connectors in use at the time, from Nokia's barrel plug and proprietary contact strip, to Sony Ericsson's clip-in connector, Apple's 30-pin dock connector and the various options (Mini-USB, Micro-USB, almost-but-not-quite Mini/Micro USB) being used by the new Android phones.

Most manufacturers quickly moved to standardize on the Micro-USB standard, as it was versatile enough through standards like OTG and MHL, could deliver enough power with USB2.0 (500mA, way up from the lowly 125mA that USB1.0 dictated), had the necessary insertion rigidity to be used for a few years. One notable exception to this was Apple, who elected to ditch their existing 30-pin dock connector, and instead make another proprietary and unlicensable "lightning" connector, with the minor fig leaf that users in the EU would get an Apple Lightning to Micro USB cable bundled with their phones and tablets.

When standards moved on, and the much more capable USB-C standard was ratified, other manufacturers quickly jumped on it, as it was much more capable with features like Power Delivery for much faster charging (and higher charging rates), massively increased bandwidth and better support for extended capabilities like DisplayPort and Thunderbolt. It's increased mechanical strength, bidirectional nature and expandability quickly overshadowed Apple's Lightning connector, and in some cases (notably their laptop line) Apple also switched to using it, but for their mobile devices Apple still held out.

So now, the EU has decided, since the manufacturers won't come together on a standard on their own, like they were supposed to, and like 90% of the manufacturers have, they'll step in and force them all to play ball, levelling the playing field and letting add-on manufacturers (for example) finally be able to innovate, rather than having to support 2 separate and disjointed systems.

Apple knew this was coming, and could have switched to USB-C at any time. They didn't, because they wanted to maintain their stranglehold (and revenue stream) on dongles and adaptors to fit their devices. Anyone who implements part of the Lightning protocol has to pay a usage fee to Apple, of course, which is also why there's no multi-function Apple dongles (unlike USB-C dongles/docks, which have a variety of configurations to suit all needs). Their claims it will "stifle innovation" pale in comparison to the locking down they've done to their own devices and ecosystem, and if they had gone along with the Memorandum of Understanding they signed to in 2009, there wouldn't need to be a codified standard. I am sure at some point in the future a better connector than USB-C will come along, and had Apple kept up their end of the agreement they signed, they would get to be a part of implementing and using that newer standard.

Apple made this bed, and now the EU is going to force them to lay in it. And I, for one, am very glad to see them getting some consequences for their actions. I'm sure the fleet of Apple masticators out there will try and cry foul, that they're just some small company that doesn't deserve to have the weight of the EU coming down on them, or some other such nonsense. Sorry, but I have absolutely no sympathy for them.

As for the whole "Maybe the UK is right with Brexit" nonsense, entirely aside from my thoughts on the matter (which are lengthy, and not at all right for this post), if you want to sell to the EU, the products you are selling have to abide by the EU's rules. As part of the EU, you get to be a part in making those rules, get to have a say in what's in them and what isn't. Outside the EU, you get absolutely no say in what they are or how they apply.

Tuesday, January 29, 2019

What I want from a CMS

Okay, I know I'm probably asking a lot, but this is my list of "want"s for a CMS.

Online Editing

Yup, I want a way to edit the site through a simple web interface. I know a lot of CMS' have this, and this would seem to be the easiest thing to get, but just bear with me.

Static File storage

I'd love for the system to be as fast and lightweight as possible. To the point that there's no need for processing when serving the site. So render it to flat files in the backend, and serve it using something simple and quick like nginx, or a simple CDN. And the static file system can be on a separate host to the admin area, for more security and less processor usage. Plenty of those "headless" systems, right?

Simple, included templating

I would really love it if this system had freely usable templates ready and built out of the box. No need to build your own site around some provided API. Set it and go, with the option for simple customizability later.

Modular design

Want to add functionality? Can do! A nice API lets you drop in modules.

eCommerce

Hey, guess what? I want it all, and that means I want eCommerce capability too. And this leads to the real problem - How do you make a storefront with only flat files?

So here's how I see it working.

The site is all flat, static files, but the optional eCommerce section runs off a tiny active shim (using something lightweight like node.js, Ruby/Rails, Go or python), loaded through JavaScript (or other means, detailed below).

When you initially visit the site, when all the static pages are loaded, one of the items loaded is a lazy-loaded script/JSON call to the eCommerce shim, establishing a session and generating an empty basket for you (or this step could be skipped entirely, with the server-side basket only being created when an item is put into it). This is then stored on the client (cookies, localstorage, webworkers, something similar). Every time you modify your basket, the eCommerce shim is updated to add/remove/alter the contents, ensuring the two are always in-sync (and verifying the details of the client-stored basket, to stop people giving themselves a discount). When the time comes for checkout, the pages are generated for the process by the shim, taking you through the necessary steps until the transaction is complete.

If the user has JavaScript disabled, the "contents of your basket" header section are instead rendered as simple HTML in an iframe from the shim, with the management being done through simple links to the shim (which then redirect to the static page required after the processing is done), and things like the basket display and editing being rendered through iframes.

Why do it this way?

It would make the site a lot simpler, and faster. Database hits would be next to none, as the vast majority of the site is static, so it could be hosted from anywhere, even a simple replicated CDN. The admin system could be hosted on a local network/intranet for security, and wouldn't have to be hosted anywhere near the main site, pushing updates to the CDN as necessary. The eCommerce shim would be lightweight, only called upon when it was needed, and allowing for simple scaling out if demand is particularly high. The system is client agnostic, working equally well on desktop and mobile, with fallbacks for situations where functionality is unsupported, including the capability of disabling completely.

It also means the site is dirt cheap to host, with a basic CDN taking most of the workload. CPU load is low, as content is static, and the static host is as framework agnostic as can be (as it doesn't require any framework, just the capability to serve static HTML). Compressibility is high, and everything can be cached without issue. The shim can be hosted in a simple system agnostic container (and automatically scaled both up to multiple containers, and back down again, as load dictates).

So what are the options?

Right now, as I see it, there are none that work this way. There are a lot of full-fat CMS'  (Wordpress/Joomla/Drupal) that try to host everything inside, making server load heavy. There are some lighter CMS' more meant for blogging (Ghost) that don't even try to touch the eCommerce side. Finally there are "Headless" CMS' that provide an admin interface (sometimes) and an API for you to write into your own site. And right now, none of these are anywhere near what I want. And unfortunately I think that means I might have to write my own. I'd really rather not, and if anyone has already invented this particular wheel (or has a good starting point for me to work from), I'd love to hear about it.

Wednesday, September 5, 2018

Inventing some virtual friends for our robot overlords

For quite a while now, I've been writing a Twitch.tv bot, that will sit in channels it's been invited into and perform some basic housekeeping tasks (dealing with spammers, keeping track of gifts, subscriptions and follows, rolling dice, notifying what channels are live and when the last time someone said something, things like that). This is all well and good, but I need something to test it against. I don't want to put in-development code into a live channel, and I don't have access (personally) to some of the features I want to code for (subscriptions and the like - that would get expensive, fast!).

So, I'm writing a kind of "test harness". It's a node.js program that will sit and wait for websocket connections. When it gets a connection, it behaves like a real twitch.tv instance, letting you log into channels, and populating those channels with "people", moderators, chatters, spammers, staff and so on. Those "people" will talk away to each other, will timeout or ban people, purchase subs (including gift subs and multi-gift subs), join, part, things like that. It won't (currently, and for the foreseeable future) do any of the web APIs, PubSub or Web Push, as that would essentially require me to re-write the entire of Twitch's backend, but this will give me something to test against, at least.

I'm wondering, however, if this will be useful for anyone else. I know other people are building twitch bots, and this will be platform agnostic (It doesn't matter what you're developing on, this little service will appear the same). And ideally when your bot is done, you just change the connection from "localhost" to "irc-ws.chat.twitch.tv" and everything just works.

Monday, August 27, 2018

New update, hot off the presses!

Wow, am I ever bad at this, huh?
Well, thanks to my MSDN/VS subscription expiring (and MS Azure finally noticing) the VM I was using to host ghost_of_leg_bot has gone away, and it's now running on my local network. To add to that, the domain ghostoflegbot.website is going to expire in about 2 weeks, and I don't have the fund necessary to renew it (Donations welcome:
). I am also working on leg_bot v2 (Current working codename "Bipedal Bot"), which is coming along. Still need to get management, follows and preference editing working, so maybe I'll be posting more about that as time goes by.

Wednesday, October 14, 2015

I want to sleep, but my mind won't shut up.

It's far too late, but my mind won't stop. And, as I have nothing better to do, and nowhere else to put it, I might as well cast my words into the void. Here's hoping this helps. I have no idea what I'm going to do with it, but apparently I need to get it down.

Trigger warning: 20 tonnes of concentrated, dehydrated shit, in a 10lb bag.

I. Am. Not. Okay.

It's a simple little lie. Someone asks "How are you?" and I reply "Oh, just fine". I'm really not. I'm a fucking mess, but I know they don't really want to hear it, and I really don't want to tell them. Telling them means acknowledging it, making it real. If I can just make everyone believe everything's fine, it'll all be perfect, right? And I'm good at lying, apparently.

I live in a house that looks like a bomb's hit it, and smells worse, and it's entirely my own fault. I am overweight and lazy and so apathetic about everything, because it doesn't make a difference. I have been living here for 6 years now, and am an illegal alien in this country. I can't work, I can't do anything, and if I get sick or hurt, I can't get any kind of coverage. The paperwork to get me legal has been filled out (at least, my half of it), but it's not been filed yet. Why? Apathy. It's a stupid amount of paperwork that has to be completed and verified, and it's not a guarantee of anything. And processing will take from 2-3 years. The fees have been paid, but wether they still have record of that or not is anybody's guess. And there's really no good reason for it, at all.

My wife, Deb, is working her ass off in retail once again, and her wage is almost good enough for us to survive on. Almost. We're not quite up to our eyebrows in debt, but they are calling, and we are occasionally having to rob Peter to pay Paul, as the saying goes. Deb is working hard, and stressed to the nines, exhausted and weary, physically and emotionally, every single day. She needs me to be her rock, that stalwart companion she can stand with and shelter from the world, and I can be that for her. I can give that impression, sure. She knows things aren't going well for me, she feels I might be a little bit depressed.

I spend every day distracting myself. I watch porn. I play videogames, badly, on easy, while sitting in an empty TeamSpeak server, and stream them to a Twitch channel that nobody watches. I write contributions to a bot and host it, just to get a little recognition. Some days it feels like I am a non-entity. If I was struck by lightning tomorrow, I can count on two hands, maybe three if I'm generous, the number of people who would notice and care. My family is 5,000 miles away, and I feel every inch of that distance some days. I have two or three friends. I act as someone else online, because I know nobody would want to know, would really care, about the real me underneath. I'm just not that interesting, not that important. Duller than dishwater.

I worry for Deb. Her health is not that good, but I can't do anything about that. She is stressing over money, as am I, and I can't do anything about that either. I can't bring money into the house to help with bills because I am currently unemployable. All I have that is in any way nice is a matter of charity or pity. I have $0 to my name, and that's not likely to change in the future. I want to be able to do something. Anything. I want to have a life, I want to make a difference. I want to be somebody, but right now all I'm doing is taking up space. I don't feel, I can't feel. Because if I do, I'm going to feel all of this shit, and it's going to overwhelm me, it's going to break me, and I can't be broken. I can't go anywhere, I can't do anything. I can't even leave, because we can't afford the funery expenses, and besides which I would never do that to Deb. I love her too much to put her through that.

And, most of all, it hurts. Deb is questioning if I actually care. I do. So very much. I care about all this shit. I want to do something, I *NEED* to do something, but I can't. Because it's too big, it's too much. I can't break it into small, bite-sized pieces and deal with it, because it's all intertwined, it's one solid lump. So I push it away, push it aside. I don't let it affect me, don't let it in.

So many things, so many ways, have I found to distract myself from dealing with this crap. And each time I've burned out. I'm burned out on life, and my brain won't let it go, won't let me sleep.

My shoes are too tight. But it doesn't matter, because I have forgotten how to dance.

And writing all this down, casting it into the ether like this? It does nothing. It doesn't help, it doesn't make a fucking ounce of difference, not a single god-damned thing. But at least my mind has quietened a little. Perhaps now I can sleep.

Thursday, July 23, 2015

Leg_Bot

Ghost_of_Leg_Bot is now dead.

Thanks to Twitch API changes, leg_bot will need an almost complete rewrite to continue working. As nobody's using the service anyway, I'm abandoning it. If you want to pick it up and work on it, the source is up on github, and if you'd like the database for the quotes or stats, drop me an e-mail.

The Ghost of Leg_Bot sent me here! What's the deal?

Well, since ufokraana elected to abandon leg_bot, and as there was evidently a need for it, I, Anaerin, have elected to update and host the bot, under the new name "Ghost_of_Leg_Bot". It's database is the final snapshot that ufokraana posted, so all your statistics and the like should still be in place.

That's cool... What about the Web API?

Well, as there's a new host, there's a new URL. currently, the web interface can be found at https://ghostoflegbot.tk - which, incidentally, is where the API can also be found for on-stream counters. The website there will be re-skinned (eventually), but for now it's a direct copy of the old site.
Update! The URL has been changed (and is reflected above). Old links will still work, however.

Isn't Leg_Bot open source?

I believe so. And I have had to make (more than) a few changes to it (to keep up-to-date with changes to the node.js library 'sequelize', as well as adding features detailed below). So I have done what any responsible open-source developer should do: Forked the repo, and am publishing my changes on GitHub.

So how are you hosting it?

The Ghost_of_Leg_Bot is currently residing on a Windows Azure Virtual machine. If you would like to donate towards it's continued upkeep, feel free to drop me a tip.

What new features are there? 

Yes, that's right, new features too!

Dynamic Counters

Counters are now dynamic - any mod can use the counter commands
  • !counter add <command> "<single>" "<plural>"
  • !counter remove <command>
to add and remove counters. Note, when a counter is removed, all the statistics logged for that counter are removed as well.
For example:

!counter add butt "butt" "butts"
!counter add warcrime "incident of friendly fire" "acts of incidental genocide"
!counter remove butt

Keep-Alive

The connection is now automatically set to keep alive, which should help with (some of) twitch's twitchiness.

Anti-Spam

The anti-spam system is now in place. If the bot is in your room, and set as mod, it'll do it's best to apply it's anti-spam rules. If anything matches, it'll timeout the user in question on a sliding scale: 4^numtimeouts. So, 1, 4, 16, 64... seconds.

Dynamic joining and parting

What's that? You want the bot in your channel? Well, there's no longer a need to hassle me - just join it's channel (ghost_of_leg_bot) and say !join. It will then join your room and start doing it's magic. If it's in your room, and you don't want it cluttering your chat with it's spam anymore, join it's channel and tell it !part, and it will leave.

Calendar Watching

On the fanstreamer calendar, but not gotten ghost_of_leg_bot in your channel? It's okay, ghost_of_leg_bot will now watch for twitch updates from any stream mentioned in the calendar. It won't join their channels, but it will be watching twitch, so you can be represented in it's !live report, and on the live page.

Statistics, by Game and by Channel

We keep track of a lot of stats. Now you can see what we keep track of, by game or by channel, at http://ghostoflegbot.tk/stats

Advice editing 

There's a lot of advice in the ghost_of_leg_bot, and not all of it is good advice. Help separate the wheat from the chaff, and vote on the advice ghost_of_leg_bot dispenses at http://ghostoflegbot.tk/advice

Barks

Got a message you find yourself often repeating? Have something you'd like said every so often? Try barks!

  • !bark add <command> "<message>" <Mod only (true or false)> [Interval (optional, in minutes)]
  • !bark remove <command>
For example:

!bark add promo "Check out my donation page at https://streamtip.com/t/anaerin" false
!bark add modsonly "This message can only be triggered by mods" true
!bark add motd "Message of the day, repeats every hour" false 60
!bark add butt "Mods are away, post butts!" false
!bark remove butt

Note: For technical reasons, editing your barks will re-set the timers of any intervals, as will ghost_of_leg_bot's connection derping for some reason.

Also, barks can now be triggered from the API: "http://ghostoflegbot.tk/api/channel/<channel name>/barks/<bark name>" Use this to give your "thanks/support/patreon etc" messages when you bring up your ending screen.

Whispers

Yes, Ghost_of_Leg_Bot now supports whispers. Error messages will be sent via whisper, and if the whisper fails for some reason it'll do it's best to find where you are and notice you there instead. To ensure whispers get through, make sure you follow the bot.

Some commands also work through whispers (those that aren't dependent on a channel), so mostly the basic ones, !nextfan and !nextlrr and !live.

Dice Rolling

Yeah, ghost_of_leg_bot does dice rolling too. Just type !roll and a D&D-style dice roll, and it will report the result to you. For example, "!roll 2d20" might get you the response "Rolled 2d20 and got 20 (1,19)"

Websockets

For those of you with an overlay, especially one in HTML (And if you don't have one, may I recommend this set of JavaScript building blocks as a starting point?) ghost_of_leg_bot now supports websockets. Simply connect a websocket to ws://ghostoflegbot.tk/ws/<channel> (For example, ws://ghostoflegbot.tk/ws/anaerin) and you will get notifications when the game changes and when stats change. For example:

{action: 'GameChanged', game: 'Creative'}
{action: 'StatChanged', stat: 'death', value: 20}
{action: 'GameChanged', game: ''}

This should help updates to the overlay come a lot faster, helping chat interactions somewhat.

Seen Tracking

Want to know when the last time someone spoke, or did something, in ghost_of_leg_bot's purview? Ask it!
!seen Anaerin

Quotes

You asked for it, so quotes are now available. (Currently disabled for cantwearhats and hpbraincase, as they have their own quote-supporting bots):
  • !quote
Returns a random quote.
  • !quote count
Returns the number of quotes available.
  • !quote <number>
Returns the specified quote. For example:
!quote 3
  • !quote bytext <search string>
Searches the quotes for something with "<Search String>" in it, and returns a random matching result. For example:
!quote bytext flying monkeys
  • !quote byauthor <author name>
Searches for quotes written by <author name> and returns a random result. For example:
!quote byauthor senstaku
  • !quote bygame <game name>
Searches for quotes that took place while <game name> was playing and returns a random result. For example:
!quote bygame Batman Arkham City
  • !quote bychannel <channel name>
Searches for quotes that too place in <channel name> and returns a random result. For example:
!quote bychannel admiralmemo
  • !quote bydate <date>
Searches for quotes logged on <date> and returns them. Note: Timezones are a thing. For example:
!quote bydate 2016-03-11
  • !quote add <quote> (author) [date]
MOD ONLY - Adds a quote to the database, returning the ID on a successful add. If the author is not specified, it is assumed to be the person logging the quote, if the date is not specified, it is assumed to be now.
For example:
!quote add This is how quotes work (Anaerin) [2016-03-11]
!quote add Not everything has to be specified [2016-03-12]
!quote add In fact, you can leave everything out.
  • !quote mod <ID> <quote> (author) [date]
MOD ONLY - Modifies quote # ID. If the Author or Date are not specified, they are not changed.
For example:
!quote mod 2 Now quote 2 has other things (Anaerin) [2016-03-11]
!quote mod 7 You can leave those other arguments off, too.
!quote mod 3 But you must have the ID and the quote text.
  • !quote del <id>
MOD ONLY - Removes quote #ID from the database. For example:
!quote del 6
  • !quote <search term>
Attempts to do an "intelligent" search, returning a random result where the text, author or game match the specified term. For example: !quote batman

AWS Support

Support for the new AWS cluster is in testing right now. Hopefully it's stable enough to use.

Tuesday, March 20, 2012

Flexget: Great program, have some ideas

Okay, I've been using FlexGet for quite some time now, and it does a fantastic job at downloading. But as I've moved to a HD setup, I'm finding there are some features I could really do with. 

Automatic PROPER stopping and removal.

As it stands, FlexGet supports when releases are corrected with a "PROPER" release superseding the broken version. But, when a PROPER is detected, it just gets added to the download queue. What I'd like to happen is that the release the PROPER is superseding is stopped and removed, and the files for it deleted. Stopping will help the proper release be seeded better, and will drop the broken release down the rankings somewhat, and it will also save on hard-drive space (and confusion) by only having one canonical copy around. Removal of files is supported in most (if not all) clients these days, so it would be a simple matter to enable tracking for that. Indeed, the Transmission (the client I use) interface has a "remove when done" flag, because the client doesn't auto-remove completed downloads. This could be (quite easily) extended.

Quality scaling and management. 

This might be a little more tricky to explain, so bear with me. Currently FlexGet supports "One" or "Many" options. What I'm proposing is that higher quality versions are kept instead of poorer ones. For example:
  • Show is released in HDTV
  • FlexGet starts downloading the HDTV version of Show
  • Show is then released in 720p
  • FlexGet starts downloading the 720p version of Show
  • HDTV version of Show finishes downloading
  • 720p version of Show finishes downloading
  • Flexget removes HDTV version of Show from download queue and deletes files, keeping 720p version.
Another scenario:
  • Show is released in HDTV
  • FlexGet starts downloading HDTV Show
  • HDTV Show finishes downloading
  • Show is released in 720p
  • FlexGet starts downloading 720p Show
  • 720p Show finishes downloading
  • HDTV version of Show is removed and deleted
This way, there's a watchable version of the show ASAP, even if it is in a lower quality. Another scenario:
  • Show is released in HDTV
  • FlexGet starts downloading HDTV Show
  • Show is released in 720p
  • FlexGet starts downloading 720p Show
  • 720p Show finishes downloading
  • Flexget stops HDTV Show download in progress and removes files
This one might be a little more controversial, as it could be seen as "leeching" (the HDTV version is stopped before it's complete). Finally:
  • Show is released in 720p
  • FlexGet starts downloading 720p Show
  • Show is released in HDTV
  • FlexGet ignores lower-quality HDTV version
  • 720p Show finishes downloading
This one is kinda a no-brainer. If you're already getting a better version, why slow that download down by splitting it's bandwidth with a lower-quality version of the same thing.

Deleting in-use files is not a problem in Linux and OSX (Handles are kept to files even after they've been deleted), but Windows is a lot stricter about keeping files in-place until all handles are closed. Regardless, however, the "deleting files" is handled by the download (torrent/nzb/http) client, and not FlexGet itself.

Of course, all these behaviours can be customised and tweaked for things like private trackers and ratio maintenance (So no removing before completed seeding, no matter what the situation, to ensure a minimum ratio for each download is met, for example), but this is something I would very much like to see included in FlexGet in the future. I hope you understand what I'm striving for.

Monday, December 19, 2011

TV is broken. Here's how we "fix" it.

I pay a content provider a subscription for a show. For instance, $0.75 for a season of House without ads, or $0.05 for a season of House with ads. $1.50 for a season of "The Daily Show" without ads. $0.50 for a season of the British show Red Dwarf, without ads. New show ideas can be funded with a kickstarter-like system. So, for example, you could donate $10.00 for a restart of Firefly without ads (Yes, I went there). If the fund reached the required amount, the project goes forward, and those who donated automatically get a season subscription for when the show airs. If it doesn't, the donators get their money back, no harm no foul. The greater the demand, the less the cost to the consumer (and, obviously, the greater the show run, or the more cost to create, the higher the subscription cost). "Reality" shows, where you call in to a premium-rate number to place your vote, could be entirely funded from this revenue stream alone, and would probably be "free" to subscribe to. This will put much more money in the hands of the content providers directly, rather than having the cable middlemen taking a large chunk, and will still keep all but the most obsessive tv-watcher's cable bill at about the same price as it already is. Watching 40-50 shows regularly (and that is a lot, TBH) would cost around $40, still less than the average cost of a home user's cable bill.

Content providers can poll me on my interests for new shows. They can also use those interests to customize the ads I get, if I am agreeing to get ads. This would make the ads both more attractive, relevant and interesting to consumers, and more appealing to advertisers.

The content provider releases episodes on their regular schedule via a private BitTorrent tracker (or similar methods) in a DRM-unencumbered format, so I can watch it on any device I choose. This releasing is done using a customized-to-the-user RSS feed (only the shows I subscribe to from that provider appear in that feed), which is secured using HTTPS and a unique "user key". Trailers and pilot episodes for new shows can also be published on the RSS feed (especially if they may match my recorded interests). Keys cannot be shared, and abuse of such will result in the key being revoked (or at least, changed, so the user has to update his client with the new key if he wishes to continue, which will make it difficult enough to make sharing of feeds like this unfeasable). Downloads could also be "Tagged", so that if a file is shared, it can be traced back to who shared it. Though, with prices like this, it's doubtful that would happen. Sharers could be penalised if they go too far (Like uploading an entire series, for example), but small-scale sharing between friends should be encouraged – it spreads the word about a show that might otherwise be overlooked, and can bring increased revenue for everyone involved. Don't punish your most fervent fans for encouraging others to enjoy what they are – they're doing your advertising for you!

News channels and sporting/current events can be subscribed to for live streaming using Multicast (preferably) or Unicast systems, in much the same way.

Cable companies can then become "content aggregators" or local CDNs, So rather than having to go to hundreds of different content providers, you can just subscribe through your cable company, and they aggregate all the available feeds for you and serve your content to you from local servers. Of course, going directly to the source is still an option, if you wish to track more directly, you don't want to support your local cable provider, or if your particular cable company doesn't carry that provider's content. Cable companies can then either add a small surcharge to the price of each subscription, or charge a "content distribution" fee for running the local cache.

Emergency broadcasts can be implemented with special RSS feed items, or signed and timestamped playlist files that point to a live stream, to be picked up and (dis)played immediately by boxes or computer systems. This is about the only part that isn't possible right now, and I shouldn't think it'd take that much work to implement.

This way, when you go to the TV, there's always something you're interested in watching available. You never miss an episode of your favourite shows, and you directly support the people who make it. You don't have to get, and don't have to watch, anything you don't want to, and you can do whatever you like with what you've paid for. It also means that niche shows don't get canned, and providers have a completely accurate dollar-based view of just how popular shows really are (rather than having to rely on a very limited number of Nielsen households), and it also means that shows aren't competing for ratings by being forced into a timeslot, giving a false impression of popularity (or lack thereof). It would also make it easier for indie TV producers to get up and running. You'd probably also have a fast addition to Google in the form of "Which provider makes this TV show".

If you're thinking this all sounds terribly complicated and difficult for (for example) your elderly grandparents to comprehend, it really needn't be. A set-top box with a friendly interface can hide all this "complexity".

So your mother turns on her TV, looks through the list of travel shows, or movies, or whatever, on her box, and chooses one to watch. The RSS and BitTorrent aspects are entirely transparent. She goes into the menu and chooses "Get more shows" and picks what she's interested in (by genre, alphabetically, by actor or director, search, whatever). If necessary, she puts in her Credit Card number (I know some elderly people prefer not to keep those details stored), and the box starts getting her new shows straight away, downloading in the background entirely unattended, ready and waiting for her to watch at her convenience.

It's not rocket science. And if she is really indecisive, she just hits the "pick something for me" button/option, which will select an (new, if she wants, or repeating if necessary) episode of a show and start playing, using the "next" button to skip shows she doesn't want to see right now (which is remembered for that playback session).

Just because it involves tech, doesn't mean it has to be technical, or difficult to use. A box with just 2 wires (Power, HDMI to TV) is dead easy to set up, and the prevalence of WiFi means it doesn't have to be any more complicated than that. Initial setup can be done with a nice, simple step-by-step wizard, which can also include initial selection of subscriptions. Trailers and ads for shows (if downloaded) can even have a "fast subscribe" option, so you just press the "Ok" button while a trailer's on and it'll add it to your list of downloads (Or present you with a "you asked for x while watching this, please confirm (and provide your credit card details)" screen after the show's finished).

So, rather than having to scroll through the programming guide on her cable box, she has a list of new, relevant TV always available for her to watch. She sits down, turns on the TV, and hits the "play something for me" button. The TV is at her command, rather than trying to fight it to find something interesting to watch.

Unfortunately, the way Hollywood, and the MPAA/RIAA/Media fatcats see the TV industry, and television consumers as a whole, this will never come to pass. They are so much more fond of their walled gardens and "Prime time slot"s that the idea of abandoning them and allowing people to live their lives not beholden to their scheduling whims is abhorrent. It would be so very nice, but I hold out little hope in it happening. And barring the emergency broadcast system, this is all completely possible to do RIGHT NOW. RSS aggregators, BitTorrent clients, streaming set-top-boxen and simple networking are all available now. This would be completely system agnostic, working equally well with AppleTV, GoogleTV, XBMC, Boxee, Plex, PopcornHour, WD TV Live and so much more.

That's what I think is "Wrong" with Television at the moment. And how you fix it. But, unfortunately, nobody that can do anything about it will listen to me, or is stuck in a non-compete, exclusive contract. This kind of scheme would effectively put the "TV Channels" out of business, or would force them to change their business model, and as they have all the money at the moment (Thanks to huge ad revenues), they really won't want anything to change. I'm sure the actual producers (Chuck Lorre Productions, Mutant Enemy, Stoopid Monkey, and morem for example) would like this system. They'd get rewarded for making popular shows, and wouldn't have the threat of cancellation hanging over their head constantly (I'm looking at you, Fox!). So I shake my fist in impotent rage, and continue to use eztv, FlexGet & Transmission instead.

Wednesday, November 16, 2011

E-Mail, DNS, IPv6, and much more...

Well, I've been having a conversation with Paul Murphy about how IPv6 is going to kill SMTP as we know it. I was arguing that it really wasn't. Yes, there is more complex work involved because you're dealing with a much larger IP base, but it's still not THAT hard to do. Here's some pseudo-code (emphasis on pseudo, it's all sketched out in Notepad, nothing more, and conforms to no known language thus far) that should give you the gist of my point.

// Connection Opens
If (dSeenHosts.Contains(sFullIPv6Address)) {
    If (dSeenHosts(sFullIPv6Address).IPBanned) {
        dSeenHosts(sFullIPv6Address.Connections++;
        Connection.Write(sConnectionBannedMessage);
        Connection.Close();
        return;
    } Else {
        dSeenHosts(sFullIPv6Address).Connections++;
    }
} Else {
    string sPartialHost = sFullIPv6Address.SubString(0,28);
    If (dSeenHosts.BeginsWith(sPartialHost).Matches.Count > 0) {
        // We've found at least one previous connection from within 4 nybbles
        // of the full address.
        //
        // We can make an assumption then that this is the same ISP as the
        // previous connection.
       
        If (dSeenHosts.BeginsWith(sPartialHost).Matches.Count > 1) {
            // There's more than one match to this partial host.
            // Hosting site? or perhaps spammer?
            float fBannedPercentage = 0;
            ForEach (var oHost in dSeenHosts.BeginsWith(sPartialHost).Matches) {
                If (oHost.IPBanned)
                    fBannedPercentage++;
            }
            fBannedPercentage = fBannedPercentage/(dSeenHosts.BeginsWith(sPartialHost).Count + 1));
            // Add an additional one to allow for this (unprocessed) host.
            If (fBannedPercentage > 0.7) {
                // More than 70% of hosts seen from this group have been banned.
                Connection.Write(sConnectionBannedMessage);
                Connection.Close();
                dSeenHosts(sFullIPv6Address).Connections = 1;
                dSeenHosts(sFullIPv6Address).IPBanned = True;
                dSeenHosts(sFullIPv6Address).BannedDate = Now();
                dSeenHosts(sFullIPv6Address).BannedTimes = 1;
            } Else {
                dSeenHosts(sFullIPv6Address).Connections = 1;
                dSeenHosts(sFullIPv6Address).BannedTimes = 0;
            }
        } Else {
            // There's only one previous connection from within this nybble.
            dSeenHosts(sFullIPv6Address).Connections = 1;
            dSeenHosts(sFullIPv6Address).BannedTimes = 0;
        }
    }
}

If (!Connection.IsOpen()) return;
Connection.SendBanner();
string sHostName = Connection.GetHELO();
bool bHostNameChanged = False;
If (dSeenHosts(sFullIPv6Address).HostName.Length > 0) {
    If (dSeenHosts(sFullIPv6Address).HostName != sHostName) {
        // That's not what you said last time.
        dSeenHosts(sFullIPv6Address).HostName = sHostName;
        dSeenHosts(sFullIPv6Address).HostVerificationFailed = False;
        bHostNameChanged = True;
    }
} Else {
    dSeenHosts(sFullIPv6Address).HostName = sHostName;
}
If (!VerifyHostNameMatchesIP(sHostName,sFullIPv6Address)) {
    // Function only returns false if it gets a result that doesn't match
    // No result is not false.
    // Not banning for this. You could, if you really wanted to be strict, but
    // The disconnection error should show up on a poorly configured server
    // And a spammer gets disconnected here.
    Connection.Write(sHostNameDoesntMatchIPError);
    Connection.Close();
}
If (!Connection.IsOpen()) return;
MailMessage mMail = Connection.ReceiveMail();
If (!SPFVerify(mMail)) {
    Connection.Write(sSPFFailedError);
    Connection.Close();
    dSeenHosts(sFullIPv6Address).IPBanned = True;
    dSeenHosts(sFullIPv6Address).BannedDate = Now();
    dSeenHosts(sFullIPv6Address).BannedTimes++;
}
If (!Connection.IsOpen()) return;
If (!DKIMVerify(mMail)) {
    Connection.Write(sDKIMFailedError);
    Connection.Close();
    dSeenHosts(sFullIPv6Address).IPBanned = True;
    dSeenHosts(sFullIPv6Address).BannedDate = Now();
    dSeenHosts(sFullIPv6Address).BannedTimes++;
}
If (!Connection.IsOpen()) return;
If (!VerifyRecepient(mMail)) {
    Connection.Write(sNobodyByThatNameHereError);
    Connection.Close();
}
If (!Connection.IsOpen()) return;
If (!RunSpamScore(mMail)) {
    If (dSeenHosts(sFullIPv6Address).SpamScore > iUserSpamScore) {
        Connection.Write(sSpamScoreExceededError);
        Connection.Close();
        dSeenHosts(sFullIPv6Address).TimesSpammed++;
        If (dSeenHosts(sFullIPv6Address).TimesSpammed > iUserSpamTimes) {
            dSeenHosts(sFullIPv6Address).IPBanned = True;
            dSeenHosts(sFullIPv6Address).BannedDate = Now();
            dSeenHosts(sFullIPv6Address).BannedTimes++;
        }
    } Else {
        If (dSeenHosts(sFullIPv6Address).TimesSpammed > 0) {
            dSeenHosts(sFullIPv6Address).TimesSpammed--;
        }
    }
}
If (!Connection.IsOpen()) return;
Server.Store(mMail);
Connection.Write(sMessageReceivedOK);
Connection.Close();

Monday, October 17, 2011

Saksatchewan, land ripe for entrepreneurship, but unfortunately backwards in technology...

A recent piece in the National Post has Saskatchewan and Alberta claiming Canada's top 9 for entrepreneurial cities, but unfortunately this pretty graph doesn't give the whole story, at least not for Saskatchewan.

Saskatchewan is, it's true, experiencing unprecedented growth, and is one of the only Canadian provinces that hasn't had an increase in unemployment. However, from a technological and ecological standpoint, Saskatchewan is really quite behind the times. There are a few indicators of this, so let's take them one-by-one.

SaskTel's monopoly makes external companies establishing a telephonic presence in Saskatchewan difficult.

What do I mean by that? Well, it seems that SaskTel is reluctant/refusing to give out Saskatchewan (306) area code numbers to anyone who doesn't have a firm Saskatchewan presence. This makes it very difficult to get any kind of VoIP service, barring the one "Open" VoIP provider operating within Saskatchewan itself and which SaskTel has a majority holding, and Access Communication's "Closed" VoIP service for access customers only. While they've not explicitly stated so, I believe this may be the stumbling block that has kept such innovative services as Google's "Google Voice" service out of Canada, despite the fact that Canada and the US share a phone system. If you look at any VoIP provider out there, it's impossible to get a large provider to give you a 306 number, because they just can't get them.

Business internet connectivity is woefully lacking.

With any other city (Edmonton, Winnipeg, Toronto, Vancouver, Montreal), there are multiple providers and multiple connection lines running throughout. It's a simple(r) matter to get a high-bandwidth high-speed line (or several) running into an office, and it's dead easy to find a whole slew of Co-Location providers for hosting all manner of services. In Saskatchewan, you have 2 options, SaskTel and Access, and neither is what you'd call "Big". All internet-related and co-lo services go out-of-province. Establishing a cloud-based service here is difficult, at best.

Home internet connectivity is poor, and the good stuff is held back by the equipment provided.

There are 2 choices of home internet provider. Access, who's service tops out at 5Mb/1Mb, and who incorporates a limit on their supposedly "Unlimited" high-speed internet. SaskTel has recently introduced it's (fantastic) 25Mb/2Mb VDSL package, which is totally unlimited in every way, but is held up by one major hiccup. The 2Wire gateways all share the same software, and that software has something of a fatal flaw – If there are more than 1,000 connections active, the gateway will start randomly dropping connections to get back under the 1,000 connection ceiling. Watching a movie on Netflix and someone decides to open WoW? There goes your streaming movie. Trying to have a conversation on Windows Live Messenger? Watch the connection bounce up and down. SaskTel do acknowledge this is a problem, and if you want to pay out of your own pocket, they can replace their ADSL gateways with a Westell modem providing basic connectivity only, and you bring your own router. The problem is that top tier package (The VDSL 25/2 one) won't work with the Westell modems. So it's 2Wire or nothing. Meanwhile in the US, with similar infrastructure, providers are offing 100MB packages for similar prices as we are paying for our 5Mb ones, and in Europe, Gigabit connections are coming in thick and fast, for incredibly reasonable rates.

SaskPower's short-sighted attitude to providing power services holds back green energy solutions.

Pretty much everywhere else in the world, power companies support green solutions. Time-of-use charging to even power loads and use excess capacity during off-peak hours, for instance, and buying back excess power from customers who have invested in green solutions like photo-voltaic cells and wind turbines. In fact, in the UK, you can "Rent your roof" to 3rd party companies, they will fit and maintain for 25 years (for free) the PV array and equipment, and take the buy-back profits from the power supplier, which you reap the benefit of considerably cheaper power bills.

Saskatchewan is perfect for wind generation – huge open flat prairie makes for long days and nights of strong, consistent winds. And on those rare days it isn't blowing to beat the band, it's hot and sunny all day long, with the "Living Skies" bathing the province in free sunlight. But without a buy-back option from the power provider, the investment in turbines and photo-voltaics is extremely heavy – you have to have enough to power your house & lands, and enough capacity to store during the leaner times, living entirely "off-grid", or you have to "throw away" the excess capacity you have during abundant times (Which SaskPower will gladly take off your hands for nothing) and eat the cost during lean times. With buy-back you make money during your abundant times, providing power back to the grid and your neighbours and offsetting the cost of the installation, and you're able to fall back on the grid when your system doesn't make quite enough.

As it stands, there are a few minor wind generating stations in Saskatchewan, though no solar stations that I am aware of. And this is a real shame. With all the open space, abundant sunlight and ever-present wind, Saskatchewan could become the power-generating capital of Canada, which would open huge employment opportunities across the province.

Agriculture has a firm hold on the economy and governmental attention, and almost nothing's going to change that.

Yes, farming and farming-related industries are big here. Huge, in fact. Indeed, I'd estimate that 55-70% of the jobs in Saskatchewan are in farming, the next 10-15% of the jobs are in service industries related to farming, 5-10% are mining and oil, and the rest are retail. There is no real technological center for Saskatchewan, and it seems that the Saskatchewan government isn't interested in any way in fostering any kind of high-tech infrastructure. Instead, they seem keen on squeezing every last dollar out of the farming industry, and draining the province dry of it's planet-killing hydrocarbons. While agriculture is a steady economic engine, and oil is a good short-term high profit business, neither really advances the province in any way, and neither is particularly attractive for outside investment or the growth of the province as a whole. As a child in Saskatchewan it seems there are only four realistic career options: Farming, Construction, Retail or Nursing. If you have the right blend of ethnicity and/or (dis)ability you may be able to meet quotas for RCMP recruitment. But any modern jobs are simply out of reach, or not worth pursuing as there's virtually no market for them.

In conclusion.

Wow, did this ever turn out to be WAY longer than I thought it would. Anyway, in conclusion, from a purely numerical standpoint, Saskatchewan looks like a great place to set up a new business. But when you look at the details, it becomes apparent that Saskatchewan is being held back, dragging it's feet on entering the modern era, and thus is destined to be left way behind. Modern businesses are in short supply, and very low demand, and what would be common services and industries anywhere else are demoted to niche markets. With a governmental and economic tendency towards stagnation and social conservatism, Saskatchewan is in serious danger of becoming even more of "the big empty space between Edmonton and Montreal". Perhaps it was a sign when the passenger trains stopped coming here all those years ago. I'm reminded of the Simpsons movie, when Springfield gets covered in a dome to protect the rest of the US from it's hideous environmental record – "[GPS announcer] Coming up on your right... Nothing...".

I'm not sure if Darren's going to read this (It's him to whom this will be tweeted first, as it was he that let me know about the above-mentioned chart), but I'll put it out there anyway. Do feel free to comment and correct anything in here that's wrong. It's written from my personal experience living here in Regina for the past 7 years after moving over from England, so it may not be entirely accurate, it is merely the impression I have received, and something that needs to be changed post-haste.

Saturday, April 9, 2011

Culinary quirks

It seems that every country has it's selection of culinary quirks. From the PB&J to Lutefisk, these dishes are often things that would make you scratch your head and ask "Whoever would have thought of doing THAT? What were they thinking?". But for all their strangeness, some (not all, I'll admit, but some) creations come under the "Now that's actually not half bad" part of the spectrum. So let's take a look at a few of these, shall we?

Starting off with the American classic, the PB&J. That's "Peanut Butter and Jelly (sandwich)", though that's the (Am.E) version of Jelly (A preserve not dissimilar to Jam, made entirely of juice, without pips, seeds or pulp of any kind, though oddly enough in this particular "dish", using Jam is just as appropriate). The mixture of dense protien, dry starch and sweet fruit (often also with a crunch, depending on the type of peanut butter used) makes this a dish of wildly clashing contrasts, which shouldn't work at all. And yet, it does, with the sweetness of the jam mellowing the gluing action of the peanut butter. And this can be extended. My wife is quite the fan of peanut butter and honey sandwiches, which is a little too much to my British palate.

And speaking of British palates, here's something entirely from my homeworld, which often has people looking at me askance: The Crisp and Salad Cream Sandwich.

Let me translate. Br.E Crisps are Am.E Potato Chips. In this particular regard, they should be "Ready Salted" flavour. Fortunately, Walkers crisps do exist in the US and Canada, though they use the brand name "Lays", and in this case "Lays Classic" chips are a perfect substitute. As for "Salad Cream", I found it here in Canada in Safeway, labelled as "Heinz Salad Cream English Dressing", with the interesting french version underneath being (translated back into English) "English Salad Cream Vinaigrette". So you take two slices of plain bread, add butter or margarine to taste, then layer on the Br.E Crisps/Am.E Chips. You need at least 3 layers, as they compress as you squash/hold the sandwich (in fact, some pre-compression may be required before assembly), and even coverage with large and unwieldy slices of potato is very difficult to manage with a single layer (Not impossible, I'm sure someone could engineer broken pieces of crisp/chip to fill the holes, but the flavour is a little too subtle with just a single layer). Then drizzle with the salad cream. This is easier said than done, of course, as a full bottle will produce a massive rush once you have gotten it going. Indeed, when the bottle is new, a much safer technique would be to insert a knife into the bottle and spread the salad cream in a thin layer on the un-coated piece of bread. Once you have creamed your sandwich, you assemble, pressing down with a flat hand on top to break down the crisps/chips into small chunks. Then you eat.

Crunchy, salty, smooth and tangy flavours and textures flood the mouth, and you know you have reached the pinnacle of sandwich nirvana. The crunching sounds continue unabated as you eat mouthful after mouthful of ingredients that were never in a million years meant to be used together, and you find it is good.

Switching gears a little, though still talking about things on bread, let's move our attention to Bitumen. Or to give it it's brand name: "Marmite".

Technically speaking, Marmite is a yeast extract, a residue of the brewing process, and many foods out there have marmite in them, albeit in it's more raw state. It's also one of the most divisive of foodstuffs out there – whole advertising campaigns have been based around the simple fact that you either love it, or hate it. So what do you do with this substance that looks not unlike road tar? Well, you spread it on bread, of course. But there is a VERY important point to make here – If you cannot see the texture of the bread through the layer of marmite, you have spread it TOO THICK! For beginners, especially, you will want to try and scrape the marmite off the bread again. the thin layer of residue that is left is more than enough for your first try. This is intense stuff, packed with flavour, and a major source of Umami, the taste that is strongly associated with meatiness. The other main source of Umami is, of course, Mono-Sodium Glutamate, or MSG.

So, what does it taste like? That's incredibly difficult to describe. Salty is certainly a contender. If you have ever had (Br.E) Beef Stock/(Am.E) Beef Broth or Consommé, imagine that, but concentrated at least a thousand times. That's what happens in your mouth, and you either love it, or hate it. Other notables in this family of substances are Bovril (Beef Extract, and not the brand of concentrated cooking stock in the US supermarkets) and Vegemite.

What else? How about "Chips on a Baguette"? Again, we're firmly in Br.E territory here, so let's expand. (Br.E) "Chips" would be (Am.E) French Fries, though "Steak Cut" fries are closer to a proper British Chip than the shoestring things the Americans typically call "Fries". And a Baguette would be a (Am.E) French Loaf or (Am.E/Br.E) French Stick, or perhaps a (Very Am.E) Submarine bun (Though those, again, aren't quite right. If you've ever seen a woman carrying shopping in a Hollywood film, she'll have a long thin loaf of bread sticking out the top of her paper bag of groceries – That's a Baguette). So, you slice your Baguette (Not all the way through, enough to make a hinge), you butter the inside (optionally), you fill the newly-made cavity with (Br.E) Chips/ (Am.E) Steak-Cut fries, and you sprinkle grated cheese over the top. Then you eat it quick as the cheese melts and your mouth exclaims in joy.

And they say we Brits have a bland palate!

Sunday, September 19, 2010

Some recording stats from my MythTV box...

Here are the current stats from my PVR. Interesting numbers to show just how much (or how little) TV we actually watch.

Recording Statistics

Number of shows:
246
Number of episodes:
12448
First recording:
Sunday January 23rd, 2005
Last recording:
Sunday September 19th, 2010
Total Running Time:
5 years 7 months 26 days 10 hrs 27 mins
Total Recorded:
1 year 2 months 8 days 13 hrs 47 mins
Percent of time spent recording:
10%
Shows Channels
Title Recorded Last Recorded # Title Recorded Last Recorded
How It's Made 1123 September 18 2010 1 CTV Toronto 679 September 18 2010
The Daily Show With Jon Stewart 948 September 16 2010 2 Space 217 September 14 2010
The Colbert Report 896 September 16 2010 3 CTV 200 September 18 2010
Law & Order 529 September 17 2010 4 Discovery 166 September 18 2010
Flip That House 485 October 18 2008 5 NBC Boston 130 September 2 2010
Star Trek: Voyager 472 September 7 2010 6 Mystery 114 September 19 2010
Star Trek: The Next Generation 395 November 26 2008 7 Comedy 95 July 28 2010
Frasier 368 December 3 2007 8 Global 72 July 12 2010
CSI: Crime Scene Investigation 366 May 20 2010 9 CTV NTV 72 September 19 2010
CSI: Miami 356 August 3 2010 10 Fox Boston 71 September 13 2010
Law & Order: Special Victims Unit 329 September 19 2010 11 CityTV Edmonton 71 September 16 2010
Law & Order: Criminal Intent 328 September 15 2010 12 CBS Seattle 70 August 21 2010
Due South 312 August 29 2008 13 ABC Boston 40 September 8 2010
Without a Trace 303 May 19 2009 14 A&E 38 August 3 2010
JAG 299 March 6 2008 15 Viva 36 June 8 2010
Cold Case 290 August 21 2010 16 CTV Vancouver 28 September 18 2010
MythBusters 262 September 12 2010 17 CityTV Calgary 21 September 18 2010
NCIS 240 September 1 2010 18 Showcase 17 July 2 2010
CSI: NY 230 June 30 2010 19 CityTV Toronto 16 September 9 2010
Battlestar Galactica 206 June 21 2009 20 Global Calgary 14 September 1 2010
Criminal Minds 197 June 29 2010 21 NBC Seattle 10 October 8 2009
House 192 September 19 2010 22 Fox Seattle 10 February 15 2010
Star Trek: Deep Space Nine 180 October 21 2008 23 Showcase Diva 9 June 12 2010
Tripping the Rift 178 September 12 2008 24 Global Vancouver 7 April 3 2010
Pimp My Ride 173 March 5 2009 25 Spike TV 5 February 5 2009

Monday, August 9, 2010

On HTML5, Video tags and Subtitles

Recently I read a rant about how HTML5 doesn't have closed caption support. This is true, but it's not so much of a hurdle as you might think. In a few hours I have worked out (mostly) a system that would work exactly the same way as the proposed extension (Barring the CSS wizardry requested – I'm not sure how to handle the requested pseudo-element selectors, so this one uses nice, simple CSS classes), all in nice, cross-platform JavaScript.

The script is copied below. I've not actually run it, or tested it, as I don't have any HTML5 video or SRT subtitle tracks to work with, but it should work in theory. And even if it doesn't, this is a reasonable starting point for building a complete system.

It's reasonably well commented, and should work just fine. I've not added any code to display the tracks dropdown, or to handle updating that part, and there's nothing in there to handle switching subtitle tracks, but that is a UI consideration IMHO. I may come back to this post and add that functionality at a later date. It also relies on being able to insert HTML elements inside a VIDEO tag. If that doesn't work, this could easily be changed to place an absolutely-positioned DIV over the top of the video, rather than drawing a DIV inside the video tag.

// Get array of videos on page
var v=document.getElementsByTagName("video");
 
// Loop through that array
for (var iVideos=0;iVideos<v.length;iVideos++) {
 
    // If the video has a "Track" element (or two)
    if v[iVideos].hasElement("track") {
        v[iVideos].tracks = [];
        v[iVideos].trackSelected = -1; // Change this to an index to change the displaying subtitle.
        // Get an array of tracks for each video
        var t=v[iVideos].getElementsByTagName("track");
        
        // Look through those tracks
        for (var iTracks=0;iTracks<t.length;iTracks++) {
            
            // If the kind is "Captions" or "Subtitles"
            if (t[iTracks].getAttribute("kind").toLowerCase()=="captions" || t[iTracks].getAttribute("kind").toLowerCase()=="subtitles") {
                
                // Add an object to the tracks array for this video object
                var iNewPos = v[iVideos].tracks.push({
                    "label": t[iTracks].getAttribute("label"), 
                    "kind": t[iTracks].getAttribute("kind"), 
                    "src": t[iTracks].getAttribute("src"),
                    "srclang": t[iTracks].getAttribute("srclang"),
                    "element": v[iVideos]
                });
                
                // Set up an async web request to fetch the text of the SRT file
                var xmlhttp=new XMLHttpRequest();
                if (!xmlhttp) 
                    xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
                
                // Store a reference to where we are, so we can work with it without having to wander through the page again
                xmlhttp.track = v[iVideos].tracks[iNewPos];
                xmlhttp.onreadystatechanged = new function() {
                    
                    // If we have a valid response
                    if (this.readyState == 4 && this.status == 200) {
                        
                        // Store the text, and fire off an SRT parser
                        this.track.webSRTText=responseText;
                        parseWebSRT(this.track);
                    }
                }
                
                // Fire off the request asynchronously
                xmlhttp.open("GET",t[iTracks].getAttribute("src"),true);
            }
        }
        
        // Set up a container for the subtitles
        var oSubtitleDiv = new document.createElement("div");
        oSubtitleDiv.style.position = "relative";
        oSubtitleDiv.style.bottom = "0px";
        oSubtitleDiv.style.left = "0px";
        oSubtitleDiv.style.right = "0px";
        oSubtitleDiv.cssClass = "cue";
        v[iVideos].appendChild(oSubtitleDiv);
        v[iVideos].subtitleDiv = oSubtitleDiv;
        v[iVideos].displaying = -1;
        
        // Set a function to update the subtitle container
        // Hopefully this doesn't fire TOO often, might want to only let this run every
        // x cycles or so, to ensure we don't bog the system down
        v[iVideos].timeupdate = new function() {
            if (this.trackSelected) {
                var track = this.tracks[this.trackSelected];
                if (track.cues) {
                    for (var iSRT=0;iSRT<track.cues.length;iSRT++) {
                        if (v[iVideos].currentTime > track.cues[iSRT].start && v[iVideos].currentTime < track.cues[iSRT].end) {
                            if (v[iVideos].displaying != iSRT) {
                                
                                // Remove the currently displaying contents
                                this.subtitleDiv.innerHTML = "";
                                
                                // Add the lines in this SRT file
                                for (var sLine in track.cues[iSRT].lines) {
                                    var oLine = new document.createElement("p");
                                    oLine.appendChild(document.createTextNode(sLine));
                                    this.subtitleDiv.appendChild(oLine);
                                }
                                this.subtitleDiv.cssClass = "cue cue" + track.cues[iSRT].cuenumber;
                                v[iVideos].displaying = iSRT;
                            }
                        }
                    }
                }
            }
        }
    }        
}
 
function parseWebSRT(track) {
    
    // Set up a variable to contain the text, with normalised line endings
    var text = track.webSRTText.replace(/(\r\n|\r|\n)/g, '\n');
    
    // Split the source into SRT blocks
    var aSRTParts = text.split("\n\n");
    
    var webSRT = [];
    
    // For each SRT part
    for (sPart in aSRTParts) {
        
        // Split it into lines
        var aSRTLines = sPart.split("\n");
        
        // Separate out the lines
        var aTimes = aSRTLines[1].split(" --> ");
        
        // Start time is the first defined
        var sStart = aTimes[0];
        
        // End time is the second defined. There may be some junk after it, separated by a space, so drop the rest
        // WARNING: This may barf, not tested
        var sEnd = aTimes[1].split(" ",2)[0];
        
        // Parse the times
        var iStart = parseSRTTime(sStart);
        var iEnd = parseSRTTime(sEnd);
        
        // Add the cue to the array
        var iNewPos = webSRT.push({
            "cuenumber": parseInt(aSRTLines[0]),
            "start": iStart,
            "end": iEnd,
            "lines": []
        })
        
        // Add the text lines to the newly-created cue object's lines property
        for (var i=2;i<aSRTLines.length;i++) {
            webSRT[iNewPos].lines.push(aSRTLines[i]);
        }
    }
    
    // Set the cues variable
    track.cues = webSRT;
}
 
function parseSRTTime(sTime) {
    
    // Split the time into parts, separated by colons
    aTimeParts = sTime.split(":");
    
    // The last part is the time in seconds, followed by a comma, followed by the time in ms
    // Easy way to sort that is to replace the , with a . and parse it as a float
    var iSecs = parseFloat(aTimeParts[aTimeParts.length - 1].replace(",","."));
    var iPosition = 60;
    
    // Loop through the parts BACKWARDS
    for (i=aTimeParts.length-2;i>0;i--) {
        
        // Increment the number of seconds, dependant on the position
        iSecs += parseInt(aTimeParts[i]) * iPosition;
        
        // Moving to the next highest denomenator, multiply by 60.
        // As the SRT only lists up to hours, this is a safe assumption to make
        iPosition = iPosition * 60;
    }
    
    // return the number of seconds total
    return iSecs;
}