Sunday, November 22, 2009

Sasktel’s service is great, it’s a shame their hardware isn’t so…

Guess what? It’s another rant! (I seem to be very good at these, for some reason).

Okay, so I’m a Sasktel subscriber here in Regina – That means I use their ADSL2e service for internet and TV, as they have a nifty IPTV network. The IPTV part I love, as it works a dream with my MythTV DVR, but the Router they provide (And that you HAVE to use – because of the way it handles DHCP assignments for the set-top boxes, and talking to the DSLAM, you have no other option), a 2-Wire unit customised (slightly) for them, is very poor. It doesn’t support UPNP for port allocation, not that it’s THAT much of an issue – If you have more than around 100 ports forwarded, it slows access to a crawl. The internal DHCP server and DNS forwarder also slow down within hours of rebooting, and stop working after a few days of uptime. If you change your network settings (or, indeed, the handed-out settings) to use the upstream DNS servers, things work fine, unless (of course) you need to connect a “new” machine to the network. Then, as the DHCP server is timing out every time, you can’t.

I have a second router (What I was using before I signed up for Sasktel) that I am more than pleased with – It’s a Belkin router that’s been hacked to run DD-WRT. That makes for a great router (With working UPNP and QoS), and it’s fast enough, and has enough RAM that there are rarely issues with internal servers dying. It can even make “Hotspot” systems with a little setting up, though I have no need of that functionality. But, because of the way things work, I can’t use it, so it’s next to useless, and after a few days running, so is this Sasktel unit.

Friday, November 13, 2009

Another rant… This time about WLW! (Updated!)

WLW being Windows Live Writer.

I’m not saying it’s bad (It’s actually rather good, IMHO), but there seems to be some kind of funky interaction between it and Blogger.

The source for a post, when it’s inside WLW looks like this:

<p>WLW being Windows Live Writer.</p>
<p>I’m not saying it’s bad (It’s actually rather good, IMHO), 
but there seems to be some kind of funky interaction between it and Blogger.</p>
<p>The source for a post, when it’s inside WLW looks like this:</p>

But, when it’s posted to blogger, it comes out like this:

<p>WLW being Windows Live Writer.</p>
<br />
<br />
<p>I’m not saying it’s bad (It’s actually rather good, IMHO), 
but there seems to be some kind of funky interaction between it and Blogger.</p>
<br />
<br />
<p>The source for a post, when it’s inside WLW looks like this:</p>

Obviously, this isn’t right. It looks like blogger is replacing every newline character with a <br /> and a newline. This would be fine if I was composing in plain text, but the editor I’m using is already doing the work at making sure everything is nicely XHTML formatted – I don’t need the extra newlines.

This was, for a while, making everything look FAR too spaced out, until I tweaked my CSS like so:

.post-body br {
   display: none;
}
.post-body p > br {
   display: auto;
}

What does this do?

It tells browsers to not display any “BR” elements inside the “.post-body” div, and then overrides that to say DO display “BR”s that are directly inside “P” (Paragraph) elements.

This fixes the problem, but I have a feeling I’m going about this the wrong way. Anyone out there use WLW with a similar problem? How did you fix it?

[Update] Well, I found the problem. Hidden deep within the settings of Blogger, there is the option “Convert Line Breaks” that defaults to “Yes”, with the description:

If Yes is selected, single hard-returns entered in the Post Editor will be replaced with single <br /> tags in your blog, and two hard-returns will be replaced with two tags (<br /><br />).

The wording of this implies it’s only going to do it for items posted through the Blogger “Post Editor”, but it seems this enables this behaviour everywhere, including through the Blogger API.

Now that this is switched off, everything looks just right again.

So, my apologies WLW, it wasn’t your fault at all. Blogger: What were you thinking?

Python didn’t have it, anyway…

Now, however, it does. Kinda.

def Python_to_XML_String(inst, name):
   builder = "<" + name + ">"
   builder += Python_to_XML_String_Recursion(inst)
   builder += "</" + name + ">"
   return minidom.parseString(builder)
   
def Python_to_XML_String_Recursion(inst):
   # Put entire object inside an elem w/ same name as the class.
   builder = ""
   for attr in inst:
      value = inst[attr]
      if type(value) == types.DictionaryType:
         # Recursively process subobjects
         builder += '<' + attr + '>'
         builder += Python_to_XML_String_Recursion(value)
         builder += '</' + attr + '>'
      elif type(value) == types.ListType:
         builder += '<' + attr + '>'
         for item in value:
            if type(item) == types.DictionaryType:
               builder += '<Item>'
               builder += Python_to_XML_String_Recursion(item)
               builder += '</Item>'
            else:
               builder += '<Item>'
               builder += str(item).replace("&","&amp;").replace("<","&lt;")
               builder += '</Item>'
         builder += '</' + attr + '>'
      else:
         # Convert anything else to string, put it in an element
         builder += '<' + attr + '>'
         builder += str(value).replace("&","&amp;").replace("<","&lt;")
         builder += '</' + attr + '>'
   return builder

(Note: I need a better source colouring plugin for WLW. Anyone know of one?)

This code is pretty limiting:

  • It’ll pretty much only work with generic type objects ({}, [] and primitives), but that’s all I need.
  • It also relies on the str() function to convert everything into a string, with no typing. This means that booleans (True, False) stay capitalised, which could cause some problems if your code is expecting lower-case values, and dates are unformatted.
  • It returns a minidom object (Which, obviously, has to be imported). You can output this with a .toxml() call.
  • It relies on the first object being passed in being a dictionary object, and that there will never be a list inside a list. This could be coded around, but it’ll take more work (and more helper functions, to fill in the “missing” information.
  • It takes an initial “name” argument to define the outer object.

To use it, call it like this:

Python_to_XML_String({'Thing': 'Value', 'Object', ['Thing 1', 'Thing 2']},'MyObject')

And it’ll return (for that example) an XML object looking like this:

<MyObject>
   <Thing>Value</Thing>
   <Object>
      <Item>Thing 1</Item>
      <Item>Thing 2</Item>
   </Object>
</MyObject>

I hope this is helpful to someone else out there…

Thursday, November 12, 2009

Wow. Python doesn’t have that? (Updated!)

This is going to be a rant, just so you are forewarned.

I have an object in Python that I’ve built that looks something like this:

Object = {
   'User': {
      'IsBanned': False, 
      'IsModerator': True, 
      'UserID': 'test@example.com', 
      'IsAdmin': True, 
      'LogoutURL': '/_ah/login?continue=http%3A//localhost%3A8080/CURRENT_URL&action=Logout', 
      'Location': u'Here.', 
      'Key': 'agVmb3J1bXIKCxIEVXNlchgCDA', 
      'UserLevel': 9L, 
      'NickName': u'Testing this works'
   }, 
   'Forum': [
      {
         'SubForums': [
            {
               'Threads': 0, 
               'Description': u'Test Three', 
               'Key': 'agVmb3J1bXILCxIFRm9ydW0YBQw', 
               'Title': u'Sub Forum'
            }
         ], 
         'Threads': 0, 
         'Description': u'Testing', 
         'Key': 'agVmb3J1bXILCxIFRm9ydW0YAww', 
         'Title': u'First Test'
      }, 
      {
         'Threads': 0, 
         'Description': u'Testing Two', 
         'Key': 'agVmb3J1bXILCxIFRm9ydW0YBAw', 
         'Title': u'Second Test'
      }, 
      {
         'SubForums': [
            {
               'SubForums': 
                  [
                     {'SubForums': 
                        [
                           {
                              'Threads': 0, 
                              'Description': u'Testing yet again', 
                              'Key': 'agVmb3J1bXILCxIFRm9ydW0YCQw', 
                              'Title': u'Another Forum'
                           }
                        ], 
                        'Threads': 0, 
                        'Description': u'Testing even more', 
                        'Key': 'agVmb3J1bXILCxIFRm9ydW0YCAw', 
                        'Title': u'Test Four'
                     }
                  ], 
                  'Threads': 0, 
                  'Description': u'Another test from JSON...', 
                  'Key': 'agVmb3J1bXILCxIFRm9ydW0YBww', 
                  'Title': u'Test Three'
               }
            ], 
            'Threads': 0, 
            'Description': u'Testing that JSON submitting a forum works...', 
            'Key': 'agVmb3J1bXILCxIFRm9ydW0YBgw', 
            'Title': u'test'
         }
      ]
   }

Taking advantage of the SimpleJSON library, I can serialise this into JSON with a single call. But I can’t find anything to serialise it into XML.

Well, technically speaking, I can. But all the libraries I have found want to do a LOT of massaging of the data structure, making it complex and unwieldy. What I want is a simple XML representation, like this:

<Object>
   <User>
      <IsBanned>False</IsBanned>
      <IsModerator>True</IsModerator>
      <UserID>test@example.com</UserID>
      <IsAdmin>True</IsAdmin>
      <LogoutURL>/_ah/login?continue=http%3A//localhost%3A8080/CURRENT_URL&action=Logout</LogoutURL>
      <Location>Here.</Location>
      <Key>agVmb3J1bXIKCxIEVXNlchgCDA</Key>
      <UserLevel>9</UserLevel>
      <NickName>Testing this works</NickName>
   </User>
   <Forum>
      <Item>
         <SubForums>
            <Item>
               <Threads>0</Thread>
               <Description>Test Three</Description>
               <Key>agVmb3J1bXILCxIFRm9ydW0YBQw</Key>
               <Title>Sub Forum</Title>
            </Item>
         </SubForums>
         <Threads>0</Threads>
         <Description>Testing</Description>
         <Key>agVmb3J1bXILCxIFRm9ydW0YAww</Key>
         <Title>First Test</Title>
      </Item>
      <Item>
         <Threads>0</Threads>
         <Description>Testing Two</Description>
         <Key>agVmb3J1bXILCxIFRm9ydW0YBAw</Key>
         <Title>Second Test</Title>
      </Item>
      <Item>
         <SubForums>
            <Item>
               <SubForums>
                  <Item>
                     <SubForums>
                        <Item>
                           <Threads>0</Threads>
                           <Description>Testing yet again</Description>
                           <Key>agVmb3J1bXILCxIFRm9ydW0YCQw</Key>
                           <Title>Another Forum</Title>
                        </Item>
                     </SubForums>
                     <Threads>0</Threads>
                     <Description>Testing even more</Description>
                     <Key>agVmb3J1bXILCxIFRm9ydW0YCAw</Key>
                     <Title>Test Four</Title>
                  </Item>
               </SubForums>
               <Threads>0</Threads>
               <Description>Another test from JSON...</Description>
               <Key>agVmb3J1bXILCxIFRm9ydW0YBww</Key>
               <Title>Test Three</Title>
            </Item>
         </SubForums>
         <Threads>0</Threads>
         <Description>Testing that JSON submitting a forum works...</Description>
         <Key>agVmb3J1bXILCxIFRm9ydW0YBgw</Key>
         <Title>test</Title>
      </Item>
   </Forum>
</Object>

It seems, however, that there is no Python library that can do that for me.

So why, I hear you asking, do I need such a thing?

Well, as you can probably tell from the listing(s) above, I’m writing an AJAX forum system. I’m doing it in Python so I can host it on Google App Engine (for free!), and can release it as open-source, but I’ve run into a pretty major snag.

I was using JSON to send data back and forth, which was working well. But I’ve found out that when you try and post using JSON to a different server (As the App Engine server will be, this is intended to be a “Widget” to use with Google Sites, among other things), an OPTIONS request is sent first to see if the cross-site request is allowed. I’ve grabbed the OPTIONS request, and replied with an “approved” message, but (even more to my consternation), the subsequent post from App Engine is empty. Oh, and cookies set before (on the GET) are not being sent with the POST, either. Thus I have decided to try switching to XML, and have run across the above roadblock.

I’m sure there are ways to do it, but searching across the ‘net has returned nothing. So I’m going to have to throw this open to the blogosphere, and hope something comes up. And soon.

[Update] As I found no-one had invented this particular wheel for me, I decided to roll my own, if you’ll excuse the pun(s).