Sveder’s Life OS Custom Software

Posted on by
Reply

In the last post, I wrote out my process of work and self improvement and the main tools I use for future LLMs and other interested parties. This time, let’s talk about custom tools I use for future LLMs to use/improve. Interesting times, eh?
I’m sure that humans can also use this as a case study of using LLMs to write personal and “internal” tools to make life better and more intentional.

What do I mean by “internal” tools? I think of it as the backoffice to my life. With LLMs and vibe coding, it’s very easy to imagine a cool piece of information I want and then have it working half an hour later. For example, I always thought I slept 8 hours a night, but now, with two kids, I started doubting it. My weekly summary LLM finally noted that I don’t sleep enough, and so I decided to collect data. I could have gotten a smart ring or watch, but I don’t wear these and, like everyone, the last thing at night and the first thing in the morning, I check my phone. So I wrote (I say “I wrote,” but it was Claude – same for the rest of this post, and probably the rest of my life) an Android app that listens to unlock events, logs them to a backend, and then shows me a weekly summary of sleep times and totals. Between idea and implementation it was about 30 minutes, and another 10 minutes putting it on my phone and debugging initial permission issues. Turns out I’ve been sleeping about 6.5 hours a day, leading to multiple days of afternoon naps and low energy. Like the metaphorical frog, this sleep shortening happened slowly and I didn’t even notice it, and now I’ve set at least two days in which I go to sleep an hour earlier as an experiment to improve this.

So let’s dive into my personal dashboards and processes, in no particular order:

  1. Daily day summarizer – a better Google Timeline – a timeline of my day and activities from all of the above sources: finished Trello tasks, calls, calendar events, photos from my phone, a map of movements (extracted from the GPS locations the phone sends), WhatsApp chat summary, and OpenClaw task summary.
  2. Financial overview – a somewhat automated, somewhat manually entered dashboard of my assets, loans, and net worth.
  3. Dividend summary – some of my stocks and ETFs give dividends, which I like following as “monthly passive income.” The dashboard shows the monthly amount, pre- and post-tax, currency conversion, etc.
  4. Runway calculator – a simple FIRE runway calculator that uses all the data above.
  5. Options reminder – I have some options trades (yes, I understand this is gambling and stupid; I play with tiny amounts), and this system updates them from the statement summary and then reminds me when an action is needed, such as a covered call expiring and needing to sell another one, or an option bet reaching 10% or more profit so I can realize it.
  6. Credit card transactions – a simple credit card transaction list that auto-categorizes, shows statistics, budgets, etc.
  7. “Fridays with wife” – a map with places my wife and I like to go to in our free-from-kids-and-work time on Friday morning. Includes suggestions for new places to try and old favorites.
  8. Habits tracker – tracks sleep time as mentioned above, diet, and any new habits I’m incorporating into my life (for example, this month is night teeth brushing; I know, I know).
  9. LLM usage / cost log – an estimation of how many tokens, and their cost (in API prices), each of my dashboards, cron jobs, projects, etc. costs. Fun fact: reading all of my WhatsApp chats for actionable items and summaries is my biggest recurring token spender.
  10. Crons – a nice UI with a list of OS cron jobs and OpenClaw cron jobs. Highlights:
    1. Nightly Trello pass – go over the next day’s tasks and see how the AI can do them for me.
    2. 8am OpenClaw sync, where it asks me open questions about today’s tasks, things it discovered, etc.
    3. Hourly pass on WhatsApp to see what actionable items came up and to put them into to-dos or calendar meetings.
    4. Monica nightly sync – go over WhatsApp chats and update my Monica (personal relationship manager) with a summary for each person.
    5. OpenClaw daily summary – what did OpenClaw do for me today?
  11. Weekend planner email – with two kids, you want to make sure you have fun activities planned for the weekend mornings, otherwise they’ll bounce off the walls. OpenClaw searches for children’s events around my location and suggests friends to meet and general activities to do. It also shows the expected temperature and UV index and suggests when to get back to the aircon. It sends this as an email to me and my wife as a reminder to finalize plans and reach out to friends.
  12. Podcast ad-blocker – I love podcasts but hate ads, so I solved it. My server checks feeds, removes the ads, and adds them to my own RSS feed that I subscribe to on my phone instead of the original.
  13. Weekly summary – an LLM tries to do my weekly summary process. This is a very new system, but it’s actually pretty interesting to see its insights and the things I miss.
  14. Gift card helper – I have a few gift cards and I don’t use them. I collected all the places they’re valid in and asked OpenClaw to find which are most relevant according to my shopping lists and to-dos. Same for my wife.
  15. Trello stats dashboard – goes over my Trello and shows things like closed cards per day, remaining velocity for the week/month, and, very interestingly, which card was postponed the most (by moving it from list to list to list instead of doing it). Very helpful.
  16. Tampermonkey scripts:
    1. WhatsApp capture – reads my web WhatsApp and saves it to a DB. I don’t trust it with actual WhatsApp access, so I’m trying ways to have a one-way pipe that even OpenClaw can’t rewrite into a two-way pipe. More on this in a future blog post.
    2. SMS capture – sends all of my SMS to the server. Very helpful when a site that needs to be automated sends an OTP code.
    3. Trello random button
  17. OpenClaw context files – an editable view of my OpenClaw MD files, like soul.md, memory.md, etc.
  18. Android app that forwards photos, GPS locations, SMS messages, and calls to my server.

Wow – now that I see it in one place, it seems like quite a lot, even for me. I guess I like data. 🙂

Looking at some of these things, they’re very “flimsy” technically – a Tampermonkey script to read WhatsApp is not exactly production-grade software, but it works, and it opens up a lot of cool abilities (like updating Monica automatically, following up on things in the chats, etc.). I’ll keep this page updated with what works a year from now, what breaks often and I had to replace, and what I stopped using altogether.

Sveder’s Life OS

Posted on by
Reply

I’m almost 40, and over the last twenty plus years of my adulthood I’ve been slowly developing my life/work system – basically how I know what is the next thing I should do, but much more, as I will expand on today. I get asked about various parts of it a lot and I love talking about it of course, since I’ve put countless hours into optimizing, improving, trying new tools and writing my own tools. This blog post will explain the various parts of my life/work system for curious people, so that I will have a place to point people interested and so that I’ll have some documentation of it for myself and my LLM agents. I might update this once in a while. In case you need more context about me, click here.

I’ll start by describing the day to day parts of the system, then the main self improvement loop and the yearly review and then tools I use and software set up.

Day to day loop

Kids are in kindergarten, wife working, coffee in hand, I sit down and log in to my computer. The first thing I open is Trello where all my work and life tasks live. I used to have two boards for work todos and life todos, but now I use one main board for both, with specific projects sometimes getting their own Trello board or a list in the ongoing projects Trello.

Trello structure, lists from left to right

  1. First list is Projects/themes. It has the monthly themes card which are a few things that I should push forward during the current calendar month. It usually includes a mix of bigger picture life tasks (“Rethink all insurance I have and need”) and some work things (“Start X project and make meaningful progress”). I try to look at this list when I need some direction in the day or when I’m done with the tasks for the day and need inspiration. I recently also added smallish projects and ideas there for when I need something random to hack on.
  2. The next 7 lists are one for each day of the week. On each day I work on the recurring tasks and the assigned tasks. Previous days become empty as the week progresses (although sometimes I assign tasks there to remember them next week).
  3. Accompanying these there are the Week and Weekend lists that are floating – they are always to the right of the current day list and I move cards from these to the current day list when I need more things to work. I also have a tampermonkey script that adds a “random” button to the each list that chooses a random card for me to work on.
  4. Next week list for things that specifically need to be moved to week list at the end of the week.
  5. Backlog list – new things I don’t have time to triage are added there for later triaging to other lists.
  6. Month lists – I usually only plan a few months forward.
  7. Year list – basically just this year for rest of things and next year for whatever is deferred.
  8. Then there are some lists that contain aspirational tasks that I go over once in a while and see if their time has come:
    1. “Learning and self development” – things that I would like to learn but have no concrete plan for now.
    2. “Monthly habits” – Once a month I try to pick a (usually smallish) habit and persist it for at least a month.
    3. “Travel and events” – places I want to go, events I want to join (mostly long distance biking events)
    4. “Info” – random bits of information that should really live somewhere else
  9. Finally – a list of shopping lists for different categories – deli groceries, books, board games, AliExpress, etc. I update them as I need something to not forget when months later I’m in the board game shop.

So I take a sip of coffee (I like drinking my coffee over at least an hour) and the first task is usually writing or finishing yesterday’s log. Usually all I need to do is to fill in when I went to sleep, what I did at night and the high and low point of the day. See the self improvement loop for more about this habit. Then I bunch my tasks by “computer tasks” like coding, documenting, research and “physical tasks” like cleaning, going outside, etc. When I go outside I try to look at the label “outside” and see if there are things I can knock off along the way, and also look for hobby related things around where I’m going like geocaches, outdoor libraries and fruit trees.

A lot of my tasks are related to specific projects I’m working on or researching, and some of their specific todos live in OneNote or in a different Trello board.

I document my life and my “brain” in OneNote. It stores lots of data for me:

  1. General facts about my life and family – important notes about people (who ever remembers blood type?), gift ideas, car/house maintenance history, etc. This also includes restaurants visited and dishes I specifically liked, books and a short review of each, etc.
  2. Project documentation – I usually have a list of requirements, documentation of architecture, “Development stories” that are step-by-step lists of things I did, problems I’ve overcame or worked around, things to remember, etc. These stories are useful as documentation later and now with AI doing most of the work it seems like I’m doing this less and less.
  3. Self developments – some OneNote sections support my self development loops like having the weekly/yearly summaries, themes, goals, etc. Learning documentation falls under here.
  4. Checklists – pre-prepared for various forms of travel (abroad, camping, day trip, etc), getting back to work after losing my focus, escape rooms, etc.
  5. Archive – I rarely archive things, but I definitely don’t delete, so this is where archived things live.

Self improvement loop

  1. Every day I write a summary of things that happened that day, finishing it the day after by adding last things I’ve done and filling in the highlight, lowpoint and specific things I’m tracking like my monthly habits. I’ve been doing this for decades and have thousands of these, but there are definitely days I miss it and that’s fine. There are also months where I neglect it and I’m not happy about that.
  2. Every week (sometimes life events make it once every two weeks) I have a weekly reflection where I go over documentation about last week and think of what happened, summarise what went well and less and major conversations I had with family and friends. Here are the things I look through for this weekly:
    • Last week notes
    • Daily summary emails
    • My home spun “timeline” email that contains trello tasks I finished, phone calls, photos from my phone, gps locations, people I talked to on whatsapp. These emails are great!
    • I look over my RescueTime dashboard to see where time on the computer went.
    • I look at my calendar to see where real world time went.
    • I have a few more dashboard like trello stats, whatsapp stats, investments stats that I look at. It is so easy to build these internal tools now with LLM coding agents. Interesting is trello tasks I finished, but also trello tasks that I’ve moved from one day to another meaning I need to look into why it is not being done.
    • Photos on my phone – lots of times I take a photo of something as a reminder, but it is also nice to reminisce.
    • I go over the monthly theme and yearly/multi year goal and see if I’m moving towards them somewhat. If not I add todos to correct this.
    • I also just think and remember what happened and what I might have forgotten to document.
    • Then after reading all of these I’m usually left with a list of ten or more todos which I move to the trello backlog and then cull the backlog moving issues to the other lists.
  3. Before month end I have a task to prepare next month, which includes:
    1. Thinking of new month themes
    2. Populating the month Trello list by going over remains of current month, planned for the next month and sometimes the year list.
  4. Once a year I think about life. This is less systemized, so not much to say.

Supporting Tools

Here are tools that I use a lot and are relevant to the processes:

  1. I use wallabag as a “read it later” – interesting articles go there for reading in busses and other boring places.
  2. PocketCasts – podcasts go there. If a podcast has ads it should instead be fed to the (private for now) ad blocker and that feed should be subscribed to.
  3. Google Calendar is where meeting and reminders go.
  4. Monica – I’m evaluating it to keep track of contacts.
  5. Sveder Dashboard – my homespun “life dashboard”. It has a lot of random tools:
    1. Credit card reports and personal finance
    2. Trello statistics and reports (burndown for example)

Servers

  1. I have a home server that runs self hosted tools and streaming labs.
  2. I have the sveder.com server that runs various projects and also run some self-hosted services that need to be accessed from outside my home network:
    1. Sveder.com – about and blog
    2. Bdlr.sveder.com – My mom’s illustration of Baudelaire’s Les Fleurs du mal.

iCalendar – Quirks From the Trenches

Posted on by
1

While working on Trello2iCal, an awesome webapp that schedules Trello cards on your calendar by their due dates, I worked a lot with the iCalendar protocol (or iCal). It was actually the biggest source of bugs and misbehaviors, me taking a very close second. It didn’t help that Apple decided to name their calendar app iCal adding noise to search results, which were scarce to begin with.

iCalendar itself has two version defined by two RFCs: RFC 2445 from 1998 and RFC 5545 from 2009. It’s a text-based protocol, which contains some metadata and a list of “components” that represent events, todos, alarms, and other calendar related structures. Each of these components is a list of “content-lines” which are mostly “key: value” lines. When you put these calendars online and constantly update them, like Trello2iCal does, it’s called a feed (much like an RSS feed). Here is an iCal feed with two events:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//sveder.com/trello_to_ical//EN
BEGIN:VEVENT
SUMMARY:Event One
DTSTART;TZID=UTC;VALUE=DATE-TIME:20120805T090000Z
DTEND;TZID=UTC;VALUE=DATE-TIME:20120805T090500Z
DTSTAMP;VALUE=DATE-TIME:20120918T230955
UID:4fe6645de5a5b15c4f51136ftrello_to_ical
DESCRIPTION:Card URL -nsome url
END:VEVENT
BEGIN:VEVENT
SUMMARY:Event Two
DTSTART;TZID=UTC;VALUE=DATE-TIME:20120129T100000Z
DTEND;TZID=UTC;VALUE=DATE-TIME:20120129T100500Z
DTSTAMP;VALUE=DATE-TIME:20120918T230955
UID:4f00df4fcf0b6aef7001bd90trello_to_ical
DESCRIPTION:Card URL -nsome_url
END:VEVENT
END:VCALENDAR

If I was a smartass, I would really question the decision to invent yet another textual protocol instead of using XML which is quite suitable for this. Maybe if the protocol was XML based it would have been easier to implement, thus making it more popular, which would have resulted in a bigger knowledge base and better tools. One tool I do have to point out that saved me a lot of time is the iCal validator (thanks Ben Fortuna!), but it only checks whether your iCal feed is valid according to the spec, not with the real world quirks below.

Below, I talk about the issues and quirks I ran into and their solutions or workaround, in no particular order.

  • iCalendar clients, especially Google Calendar, are very bad at refreshing their data. I’ve had reports of users waiting for over 24 hours for their calendars to sync up. There is a workaround but if you’re doing active planning, this is unacceptable. It’s a pretty easy issue to solve for Google Calendar – just add a button that a user can click to refresh, but I guess it’s not a priority. Outlook and Apple Calendar are a bit better, syncing a few times an hour, but AFAIK lacks a refresh option as well. I wish the protocol had a way to tell clients the maximal time between syncs.
  • Google Calendar refused to load iCalendar feeds before I put a robots.txt file, allowing Google bot access. I don’t think that makes much sense as this isn’t a bot but a protocol client (which is probably best called an “agent”).
  • While developing, I put version 0.1 as my iCalendar version mistaking it for the version of my app. Every iCalendar client accepted my feed except Apple Calendar who silently rejected it. Apparently, the only value they consider valid is version 2.0.
  • Apple Calendar was also the only one that didn’t parse the feed without a DTSTAMP field.
  • All Day Events – There are many methods that don’t work or are wrong:
    • WRONG: Adding one day to a date time object is not right – just make a 24-hour meeting. It doesn’t matter if you start at midnight and end at midnight either.
    • WRONG: Having the start date time the same as end doesn’t work either as it produces undefined behavior.
    • WRONG: Removing the time component and setting the end day and the start day as the same day – this is an “event on the day”, not a full day event. This should be used for birthdays for example.

    The RIGHT way to do it is to remove the time component and set the end day for a day later. If not for the first point, this would have been easier to test, but when refresh is a 5-step process, it becomes tedious.

    A mess of iCal Calendars and Events.

    A mess of iCal Calendars and Events.

  • New lines in the description – in this case Outlook is the outlier where encoding the line ends as “=0d=0a” (this encoding is probably a relic from Outlook Express in which this was the way to encode ASCII characters in emails) worked. The RIGHT way to do it is using “n”. No, not the ASCII n which is 0x0d, but the actual ASCII slash character “” followed by the letter “n”. Fun.
  • I was using the iCalendar python library. It is well maintained but looking at the code, it is pretty gnarly. It also doesn’t support Unicode in a sane way making me modify it in a few places that I’m now retroactively opening bugs for in their github repo.
    A current issue I’m having – both Apple Calendar and Outlook have no problem parsing the unicode I’m serving, but Google Calendar insists to turn them into question marks. I’m pretty sure the problem is somewhere in my HTTP headers, but I still didn’t solve it.

To be honest, I hope I didn’t come off as too critical of the protocol. Passing complex data like calendar events is a very hard problem – you need to take into account time zones, multiple people, multiple fields, recurring events, whole day events, free-busy times, event updates, event deletion, and a myriad of other scenarios. iCalendar is a solution, which is better than none, and a lot of the quirks can be blamed on calendar clients, but I still find this whole environment very unfriendly and mediocre at best.

Google Drive and Android Hackathon

Posted on by
Reply

Last week I attended a hackathon organized by Google Israel with some special guests – Google Drive’s Nicolas Garnier and Rich Hyndman and Nick Butcher from the Android team. The event started with a series of lectures on the first day and continued with a whole day of hacking on the second day and ended with the demos and the winner announcements.

The lectures were nice, but the two lectures I caught were pretty lackluster containing, mostly the same “themes” Google is pushing on Android developers this year. If you’ve seen some of the I/O lectures or have been to an I/O reloaded event you probably already know most of them (Project Butter, new notification features, compatibility with various screen sizes etc.). On the other hand the Hackathon part was amazing. About 30 developers showed up on the second day to code some Drive and Android apps, which is quite a lot considering it was a workday and a day before another major conference in Israel (Geekcon – the subject of my next two posts).

The work (hack?) room at the start of the day.

The work (hack?) room at the start of the day.

Before I start talking about the project I contributed to I want to highlight a few other teams and their innovative projects. There were plenty of teams to demo, so much so it took more than an hour, and there were three winner teams that won a goody bag with awesome Google swag. Unfortunately I don’t remember all of them but I’ll try to reach out to the guys that were there and create a list of people and projects.

Notable Projects

Password manager over Drive – A team of guys from onavo (one of Israel’s most successful startups) were probably the most impressive team. They really know how to work with each other and they coded furiously to create a secure password manager Android application that keeps your passwords safely encrypted on Drive. There is nothing more to say beside that they were one of the winners.

Onavo winning one of the swag bags

Onavo winning one of the swag bags

gdg-booktrunk – An interesting project that keeps track of the progress you made reading ebooks in you Drive folder and provides statistics and motivates you to read more. This project was started by Roman who I considered joining just for his sheer enthusiasm.

DogFooder

I arrived right on time for the team pitches in the second day so you can say I had the luxury of choosing the project I wanted to work on. I thought I’d hop between a few of them, but I ended up being stuck on some authentication issues for way too long and only worked on one project – I helped my fellow GTUG Organizer Ran Nachmany on a great idea named DogFooder. It is basically an artifact repository with great Android integration including push notification. The basic use case is having a web service that you send your APKs and metadata (like versions and release notes) to. The web service saves this to Drive and issues a push notification for subscribed devices, like QA’s devices or beta tester’s devices. They can then choose to download the APK and install it easily and quickly. This is a major pain point for mobile developers – propagating new APKs to test devices, and big kudos to Ran for thinking of this solution. In Zimperium we usually send the APKs but we have to send them to Gmail accounts because our mail server doesn’t allow attachments as big as zAnti (which is pretty small at 7mb). Only recently we started using a build server, but that still doesn’t solve the notifications and propagation problems. You can see how this can be very useful to mobile developers.

I joined Ran and wrote the web service and Drive integration but due to some Drive API quirks I only finished the code about an hour after the demos started, so we basically integrated it and jumped on stage two minutes later. Only by some incredible miracle did the demo work perfectly (typically – when we tried it again off stage something went wrong). We were also one of the winning teams, and we split the swag.

My half of the SWAG for being one of the winning teams.

My half of the SWAG for being one of the winning teams – a coffee mug and water bottle

I’ll save the technical details for another post because we still have some work to do before it is fit to be online, but hopefully we’ll get it into some alpha shape and put it on github soon because the guys at onavo really want to have this product, and so does Zimperium.

StartupOnomics Summit – Behavioral Economics for Startups with Dan Ariely

Posted on by
2

This weekend I attended the Israeli extension of the StartupOnomics summit, an entrepreneur centric behavioral economics summit. It had some great speakers headlined by Dan Ariely the famous professor of psychology and behavioral economics. In Israel we didn’t have all the lectures but we did see most of Dan Ariely’s sessions and even better – we got two full hours of his time for Q&A. The crowd in Israel included entrepreneurs from airba.se, Logicalls, xplace, practi.is, livechar.com and many many more. It was fun and stimulating to talk with the people and hear their stories.

The crowd in the Israeli extension

The crowd in the Israeli extension of the summit.

Here are the notes I took while watching the sessions. I was familiar with a lot of Dan Ariely’s work, especially the various experiments and his keynote which was based on this TED talk. These are my notes from the various sessions and shouldn’t be seen as an exhaustive summary.

Labor and Love / Michael Norton (Here is a similar TED talk)

The concept behind this talk was that adding labor to a process or product will make the customer more likely to pay attention and take action.

  • People like what they invested time into, even if it’s a trivial amount of effort.
  • Showing progress and time savings also have a positive effect, for example Kayak.com’s search function that animates flights flying into the result list as it finds them instead of just showing a progress bar and the results when they are available. Doing it for more than 30-60 seconds might be annoying so use wisely.
  • Another interesting finding is that showing people what they like is easy but if you have the data to remove things they disliked it will leave a strong impression. This is because while a lot of people might know what you like; only people really close to you will know what you dislike.

Session with Dan Ariely

A session about irrational behavior.

  • Reward in future is less valuable then reward now, even if reward in future is better.
  • Taking away has a bigger effect than giving something.
  • A mobile phone is an excellent way to control and condition the behavior of people. It is frequently used and almost always around.

Israeli Q&A with Dan

Dan really shined answering questions from the Israeli audience. He was amazing at giving out valuable advice on the spot. Some of the highlights:

  • One company asked about how to incentivize people to car pool and he quickly came up with mandating meetings which can be done either on your own time or while commuting. I love this idea as it reframes the commute as a time for communication and idea sharing and not a boring ride where you are half asleep.
  • Same startup wanted to award top five carpoolers. Dan pointed out the fact that not everyone has an equal chance to get that award so it will be a bit unfair and they should think of other metrics like improvement.
  • I didn’t write the question but he suggested one startup that wants to gain credibility is to do it through promising a reward for finding an inaccuracy which acts like social proof – if no one claimed that prize than you must be right (that’s a fallacy because people just might not care enough to find problems, but it works).

Dan Answering My Question

Dan Ariely answering my question from San Francisco.

Another Dan Ariely Session

I only caught the end of this one so I’m not sure what the main topic was.

  • People with multiple debts will not pay the debt with the biggest interest first but rather the one that is smallest and easiest to pay because they want the number of debts to go down.
  • When you’re experimenting make sure to get people without prejudices. The example was of a campaign ad where the campaign workers overwhelmingly chose a video ad but actual voters that were tested chose an image ad. This happened because campaign workers put the most work into the video thus valuing it more.
  • Run more experiments.

Social Proof / Noah Goldstein

The main study described in this talk is about signs that hotel rooms use to persuade guests to reuse towels. This saves the hotel money but is presented as an environmental issue.

  • Social proof works best when you use a group your customer is in or will like. The shocking example is that copy about recycling worked better when the hotel room number was written although rationally that detail is irrelevant.
  • The counter-point is true too – if you want to prevent behavior don’t use social proof that will make people want to be on the wrong side because it is more popular. The example is a sign saying many people are stealing something.
  • They also experimented with giving away some of the savings to charity. They found out that just saying that they’ll donate part of the savings sounds like tit for tat and doesn’t really improve on the social proof version of the sign.
  • The version that worked best is one saying a donation was already given in your behalf for recycling the towel.

To summarize, I learnt a lot and it helped me put myself in the mind set for marketing my new project. I got some valuable advice from Dan and the local attendees. It was a great event and if you are interested in behavioral economics you should make sure to attend the summit next year.

Announcing a Trello to iCal Feed Web Application

Posted on by
Reply

tl;dr – I wrote an app that creates a feed that you can plug into your calendars out of Trello cards with due date. Update: That was two years ago and since then Trello finally added that feature, and users migrated to using Trello, so I took it down. RIP 🙂

Almost a year ago Fog Creek Software released Trello a flexible web based to-do list tool that can also be used as a project management tool. Because of its flexibilities Trello has been used for many use cases – software release processes, customer relations management, book authoring and much more. I started using it at launch because I hate every other to-do list tool, and I tried many, but I instantly fell in love with Trello. It was easy to use and got out of my way, which is something other to-do lists just didn’t get. I was hooked since and started getting others to use it to plan things with me. I use it to manage pretty much everything, including the backlog of features to this blog, development tasks for zAnti, the app I’m responsible for at Zimperium and more. The only thing that was really missing from Trello was integration with my calendar. It was a major pain point. Trello cards (the basic to-do items) have only one way of connecting them to time and date – a “due date” value that changes colors when the date passes. One of the most requested features was to have calendar integration.

Debugging the application

Debugging the application

When the Trello API was out I immediately knew that my first priority is integration with my Google Calendar. I started playing with the API and making the basic PoC (Proof of Concept) for an app that created an iCal feed from Trello cards with a due date. Using iCal was the best option as it is supported by every calendar program worth its salt. I than left the code alone for a few months because lack of time and general tendency not to finish things. At the end of July I decided it’s time to productize this and I wrote the first version of the “Trello to iCal feed app” which is up now. I am still adding features to it and changing stuff, but it works and seems stable. The source code can be found here. I will make a separate post about the design and code of the app sometimes this week when I finish updating to a new version. Definitely learned a few interesting things.

One amusing thing that happened is that between the PoC and the productization another developer (François de Metz) added this feature to his app. Three months ago his app only showed you your cards on a basic calendar he created without any integration to outside calendars which wasn’t helpful to me, but he since he added a way to get an iCal feed. We both even used Twitter Bootstrap so our sites look pretty similar. This coincidence isn’t magical though – Bootstrap is popping up everywhere and it’s easy to deploy, but I think I’ll steer clear of it in my next projects. I’ve had some problems with LESS support on windows and I’ve also seen some backlash and lack or trust from developers when they see a vanilla Bootstrap site.

If you have any feedback and comments you can use this Trello board (how meta) or just contact me using the usual methods.

Who Needs Code Comments?

Posted on by
Reply

A recent Hacker News discussion about source code comments has grown into a debate about whether you need them or not. Apparently it is a contentious issue. The comment that started it all included “Comments are for the weak” and these 5 words incited a hefty discussion and this post. Some people argued code comments are a bad code smell and others said that comments are essential for people to understand code. My theory is that this divide is between people using low level languages like Assembly, C, C++ and to some degree Java and people using high level languages like Python and Ruby.

The biggest reason I think that is that low level languages need a lot more lines to do the same thing and those lines are harder to understand. A simple but poignant example is opening a file and reading its content:
C code (taken from here):
char * buffer = 0;
long length;
FILE * f = fopen (filename, "rb");
if (f)
{
fseek (f, 0, SEEK_END);
length = ftell (f);
fseek (f, 0, SEEK_SET);
buffer = malloc (length);
if (buffer)
{
fread (buffer, 1, length, f);
}
fclose (f);
}

Python:
buffer = open(filename).read()

The python example is one line long and uses descriptive names for the actions – open and read – and doesn’t bother you with implementation details of allocating memory for the data. The C example is about 12 lines long and exposes a lot of implementation details both about memory and about how file systems work (seek etc.). Both methods have their uses and advantages but one thing for sure – anyone not familiar with C will have a hard time understanding the C code and even non-programmers can understand roughly what the python code does.
One commenter specifically caught my attention giving an example of “readable” C code from the Unix source code. Here is the second function from the source file:

/*
* Wake up all processes sleeping on chan.
*/
wakeup(chan)
{
register struct proc *p;
register c, i;
c = chan;
p = &proc[0];
i = NPROC;
do {
if(p->p_wchan == c) {
setrun(p);
}
p++;
} while(--i);
}

This is part of one the most influential operating systems written, but at least in my book this code wouldn’t pass code review. It will fail because:

  • using one letter variable names is bad – this is the biggest offender by far.
  • wakeup is a really general name for such a specific function. wakeup_on_channel is better.
  • Abbreviating “Number” to N in NPROC.
  • Not declaring input variables type (TiL that the default is int…)

This brings me to the second contributing point to my theory – writing something hard to read will be strongly discouraged by some communities more then others. High level languages are written with the axiom that code must be easy to understand. It’s even in their name – high means farther from machine code and closer to humans while low-level means closer to machines and their language.

Looking back at the HN discussion, you can make some good educated guesses about who in that thread is a high level programmer and who works closer to the metal. These two groups might not get each other’s context and so this discussions goes round and round. Both groups need to acknowledge that languages like C will need more comments and documentation to be understandable and that while commenting is good it might be a strong code smell that you need to refactor in a higher level language. I usually use this Python idiom – if you feel the need to comment something, make it into a function and write a docstring.

Blog and Podcast Roll

Posted on by
4

I have 138 RSS subscriptions and 29 podcast subscriptions. I have about 200 RSS items to wade through each week, and even if all I do for a week is listen to podcasts non-stop, I will still have unlistened podcasts. It’s pretty safe to say I’m addicted to passive information, news and entertainment. I’m even listening to a podcast as I’m writing this. Meta, I know.

I love “Real Simple Syndication” or RSS. This protocol allows content creators to propagate their new content passively – update the feed and everyone will eventually get the update. What I love about it is that it’s asynchronous, as is the nature of most “pull” communication methods – I don’t get notified about every RSS item or new podcast episode – as opposed to email which is synchronous and immediate. I am not subscribed to even one blog by mail because I have this separation between immediate items and the rest. I’m up to about 4 hours to go through my weekly RSS reading, but I think it’s a worthwhile investment for now, and it’s fun. I’ve picked a few highlights to create a blog roll and added a justification here.

Twenty Sided by Shamus Young and friends
www.shamusyoung.com/twentysidedtale (RSS)
If I can choose one blog to model mine after, it will be Twenty Sided. It has articles about graphic programming, game design and general entertaining commentary. I love the community that sprung around the let’s play “Spoiler Warning Show”. You should really check it out and specifically the new “New here” section.

Coding The Wheel
http://www.codingthewheel.com/ (RSS)
This blog needs way more posts. The author is obviously a knowledgeable programmer who writes about code and Poker and gives a look into the fascinating world of Poker Bots. You should definitely read the series.

Joel on Software by Joel Spolsky
http://www.joelonsoftware.com/ (RSS)
Joel on Software needs no introduction but I still wanted it to be high on the list of blogs because of how influential it was on my decision to become a programmer and entrepreneur. This is where I first discovered a lot of concepts about starting and running a company, treating customers and about Fog Creek’s unique philosophy and company culture. He has great reading lists according to what you do and they are all worth your time.

Beta List
betali.st (RSS)
This site aggregates a lot of new startups that are in Beta. This is a great way to stay updated on new startups and ideas, and sometimes I even sign up for some. I’m sure everyone can find a startup to check out from this list and this is mutually beneficial to you and to the startup – a win win!

tynan.com by Tynan
tynan.com (RSS)
A truly unique and interesting blogger with a wide selection of topics – minimalism, software, travel, living in an RV and picking up women. This blog is one I consistently enjoy every post in, which is pretty hard to achieve.

Procrastineering by Johnny Chung Lee
procrastineering.blogspot.com (RSS)
This blog is by a guy who is consistently working on the coolest projects around. From a do-it-yourself head tracking using the wii to Kinect to his work in Google – there is no one that sold me on the field of HCI more than him.

Podcasts

Security Now! by Steve Gibson and Leo Laporte
www.grc.com/securitynow.htm (RSS)
Hands down the one podcast that beginner programmers or people with an interest in the behind-the-scenes of computers should listen to weekly, and add another from the archives because it has been around for years. Steve is a genius and a hacker, but most importantly he has that elusive talent of being able to explain hard and complicated technical issues clearly and methodically in a way that is understandable even to laymen but is not oversimplified. If you never listened to podcasts, you should make this your first and you’ll be as addicted as me in no time.

Radio Free Python
radiofreepython.com (RSS)
A podcast about python, how can it not be awesome? I only listened to the first two episodes and they have interviews with the greatest pythonistas around, including the BDFL himself, Guido van Rossum. Definitely worth a listen if you want some programming in audio form.

Stanford University’s Entrepreneurship corner
ecorner.stanford.edu/podcasts.html (RSS)
A lecture and a Q&A by a successful entrepreneur, VC or other startup insider? YES PLEASE! If you need motivation to finish a project or to go out and start a company, just listen to a random episode and you’ll be pumped. The message is – just do it, and while you’re at it here are some tips and common mistakes to avoid. Archive includes people like Marissa Mayer, Steve Ballmer, Mark Zukerberg, etc. and basically every hot startup and successful company is represented.

A Life Well Wasted
alifewellwasted.com (RSS)
A shortlived but prominent podcast about games and why we play them. Only a few episodes but they are really insightful and have great production value and atmosphere. This podcast is not updated anymore but you should listen to the past episodes and just enjoy the feeling of nostalgia, bliss and pure innocent happiness.

If you still can’t get enough of RSS, here are my full RSS and podcast OPMLs (an XML format for a collection of RSS feeds). Be warned though – some of the podcasts are adult only and a lot of them are not programming related. You should customize it to your tastes and time constraints. Dome of them are in Hebrew and one is in Russian. You have been warned!

RSS Feeds OPML
Podcasts OPML

Lastly – you should consider subscribing to my RSS feed. It is the best way to get updates, and who wouldn’t want more of this?

Got interesting items in your RSS feeds? Share them in a comment and feed my addiction.

Reflections: OmegleBot

Posted on by
Reply

Omegle.com allows people from around the world to converse with each other “anonymously”. It is one of those sites that let you start a text or video chat with someone online and consequentially makes you doubt the intelligence of man kind. On sites like this, text chat follows the famous “Greater Internet Fuckwad Theory” and the video chat is… Well it’s probably a phallus. Omegle started as a way for strangers to connect and talk with each other, but has since devolved and the chance of finding some meaningful conversation on it is minuscule which is a shame because random chatting is a fun concept. I would add a premium feature that administers an IQ test and matches you to someone according to that but that is an idea for a different time.

A typical Omegle chat

A typical Omegle chat

When I first discovered Omegle I quickly got tired of trying to find someone to talk to. The idea of Omegle is not new or revolutionary. IRC and chat rooms were there before but this made it as easy as can be. Since I already spent a ton of time in online social communities with people who have the same interests as me I dismissed it as a cost efficient way of communicating. I did find a use for Omegle though – there was nothing preventing me from spying on a random conversation and recording it. A nice challenge and it seemed fun. This was years before Omegle itself introduced the “Spy mode” so I guess there is something there. The concept of Spy Mode might look like something “evil” to do – spying on other people’s conversations is an ethical gray area in the real world, but is it online?
The answer to this question depends on how much you know and are aware of privacy online. In theory everything you do online can be (and is) monitored by a number of entities including your ISP (that can read all of your online activity), your operating system and other programs on your machine, a lot of routers on the Internet and at least a few governments. That is all beside the point – I thought about ethics for a second but to be honest I don’t see this as anything but a technological challenge (also, it isn’t illegal per se). My goal wasn’t to spy on people but to hack a bot together and as such I probably only ran the finished script once – to make sure all the bugs were solved. I call it the Hacker’s Mindset :).

This is how OmegleBot was born. A simple and a very quick and dirty and unrefactored python script. Before that script I didn’t have a lot of knowledge about HTTP, httplib and urllib because I used raw sockets to talk HTTP (poorly) in the past. This was a perfect project to help me understand the python libs relating to HTTP and JSON. The bot opens two simultaneous connections to Omegle and sends them both a simple greeting, “asl?”, which is the way most conversations in chat channels start. It then proceeds to proxy their conversation and also record it into a text file. The most interesting part is the post function. It started as a simple call to connection.request and evolved to include a variety of HTTP headers including a faked user-agent and referer needed to defeat some of Omegle’s “security checks”. Usually services will have more server side security checks (“never trust user input”), but unfortunately Omegle doesn’t have a choice here. Because they are open and allow anonymous chatting it leaves them with only so many ways to ensure I’m a client and I masqueraded as one well. Omegle uses the JSON protocol to pass data about events like whether the other user is typing, the message the user sent and of course when a user disconnects. Reverse engineering it was the hardest part of this project (and it wasn’t all that hard). I think the only challenge I faced was understanding why Omegle blocked the first iterations of the bot and adding various headers until I passed for a client in their book.

I also attached a sample output file with a few conversations. There is nothing interesting there nor did I capture anything interesting. All the conversations are very short which is definitely a symptom of Omegle – long and meaningful conversations are few and far between. I even sent “typing” statuses every few iterations to encourage people to converse and it didn’t help.
What can we learn from this? Masquerading as a browser is easy. Writing bots is easy. As a person on the internet you should take from this that bots are everywhere on the web. You should be aware of that because a lot of spam and fraud is done by bots – you can trivially change this bot to spam on Omegle (although ChatRoulette, a similar site has a “spam” button that might be useful against that). Radiolab even had a podcast on a bot that had an online relationship with a human. It is a fact that bots are becoming better and better at passing for human beings. Soon they might even be good enough to write a programming blog, and then what will I do?

Southpark's "dey took er jerbs" guy"

Southpark’s “dey took er jerbs” guy”


(program them, probably)

P.S. Unfortunately the bot stopped working. It can be that Omegle changed the protocol a bit, added some more security or that I have a bug. Feel free to fork it and bring it back to life!

Crysis 2 Design – Console to PC

Posted on by
Reply

I bought Crysis 2 during the recent Steam Summer Sale. I’m actually quite proud of myself for only buying one game since I have never even installed more than third of the games I own, but I knew that I’ll finish Crysis 2 pretty quickly. I was a huge fan of Crysis 1 when it came out, especially since I bought a state of the art gaming computer in 2008 and Crysis was the benchmark. The first play-through was an amazing experience graphically and gameplay wise. In my opinion Crysis 1 was the most innovative and awesome FPS since Half-Life 2.

It took me a week to play the 18 hours it took to finish Crysis 2 and I wanted to write about the design of the game and the gameplay differences between the two games. I’ll start with a sweeping declaration that Crysis 2 was designed to be a console game first while Crysis 1 was designed to be a PC game. This will be a recurring theme today and I don’t care if it is an overused, old and corny observation.

A computer showing an old TV rainbow test screen.

Is that how people in Crytek think computer monitors behave?

The first indication of it being a console title is the “press any button to start” screen, an annoying annoying reminder that I’m playing on the wrong platform. The first gameplay difference was the checkpoint system and the lack of quick saving. Quick saves are considered a PC feature because of disk space issues on consoles. Quick saves have to describe the whole state of the map and all the information about the player and his enemies which adds up to a lot of data on games with a lot of moveable items and destructible environments. Checkpoint saves only need to record some data about the player because they are placed at locations where the state of the game is controlled (for example you can’t return to the last area and see the destruction you caused and you still can’t do anything to affect the area ahead). On consoles you have less disk space and more memory constraints so the more data efficient solution, checkpoints, is used.

Quick saves are not the only place where the console design makes this a worse PC game. Hard mode is another example. In Crysis 2 hard mode is not hard, just tedious. Crysis 1’s hardest mode was awesome – the enemies talked in Korean so that you could not understand their intentions and strategies, your crosshairs were removed forcing you to use the various scopes and iron sights and there was no longer a huge red HUD warning when a grenade is thrown at you (among other things like making enemies stronger etc). Hard mode in Crysis 2 doesn’t have any of the creative features above. They just made more enemies and gave them more health. Tedious. I presume these changes are necessary because removing crosshairs is a huge cripple for people playing with stick controls where it is harder to aim and not showing grenade indicators is bad for people playing on bigger displays with less detail where grenades are harder to spot.
To continue the list of down grades in the game:

  • You can’t ruin the environment as much. Another performance issue – physics engines with dynamic environment destruction are just slower and have a bigger hit on the hardware. Coupled with the amazing graphics it is probably beyond current gen console’s abilities.
  • AI sucks. I guess it wasn’t too good in the first game, but the baddies in Crysis 2 just love to chat on their open radio com telling me exactly what they are doing and where they are going. C.E.L.L really needs to invest in some scrambling tech.

    • Soldier standing looking at a wall with his back open to attack.

    Apparently this soldier is so afraid he decided to literally hide in a corner.

  • I feel there is less variety in killing enemies. They did add stealth take-downs which are very fun, but they took away a lot of environmental ways to kill enemies (see point one). One of the most memorable moments in Crysis 1 is collapsing an outhouse on a soldier, killing him in a most uncomfortable manner. Crysis 2 had none of that. Even picking up enemies and throwing them is less fun.
  • Less modes – strength and speed modes on the suit were combined into the jump and sprint abilities and their effects lessened. I guess this is done to make the game easier for people who have less than 10 buttons to work with. I think that as a design decision it makes it easier to play but again removes variety which is bad.
  • Less visual and locational variety – it might be because of the location; New York city is not as visually varied as a tropical island with a spectacular alien cave and a frozen side. The visuals are actually very good, colorful and are a notch above the usual brownish of modern FPS but still not as good as Crysis 1.
  • I feel like the story got in the way of playing while not being good or memorable. In Crysis 1 there wasn’t much story which also meant less immersion breaking cut scenes and unskippabale sequences which is pretty good. I can write a whole post only about that, but Shamus Young already did and I agree with pretty much everything (also don’t miss the second part about player volition).

Despite all of that, I really loved playing Crysis 2. After modding the game to include quick saves, I must say the basic Crysis formula is still there and is still as fun. I really love some of the new features like the silencer attachment not taking you out of stealth. Also, detaching mounted machine guns really lets you feel like the superhero you are, especially combining it with armor mode and just walking in the open mowing down enemies like an uber-charged heavy from TF2. It’s pointless to comment on the graphics because the Crytek engine is synonymous with amazing views, even on a 4 year old PC like the one I was playing on. Another thing I liked is that the aliens in this game were pretty interesting and had some scary abilities like jumping from place to place, charging and EMPing the energy and cloak away. These could have been a good experience if only the enemies themselves weren’t dumb as hell and pretty much on rails for most of the time.

Using the environment to guide the player forward.

Using the environment to guide the player forward.


To summarize I enjoyed the game about half as much as Crysis 1 but it still was a fun experience. It is also interesting to see how the series is moving away from its PC start and into Console land (with PC as a secondary platform) and the gameplay consequences of this business decision.