Development Roadmap – Q2:2015

It took me a while to put together this quarter’s Development Roadmap without giving away too many details or secrets about some of the upcoming work in this project.
I can now discuss a little bit about the crawler bot, and something new going on with the rating system. I want you all to know that I am very alive, and this project is still active.

Q1 Setbacks

Last month I got everything stuck into a bit of a migration phase, so I’m still moving some stuff over to a new server. One of the problems that exists on the previous server is that the crawl bot isn’t running as efficiently as it could be, so I’m really pushing to get version 2.x of the crawler done before I complete the migration process once and for all. So on top of keeping up on support requests, keeping up on development and keeping up on maintenance of the servers, things have been moving along slower than I had hoped. Some things expected to be done last quarter were not done, and for that I do apologize.

Q2 Expectations

I have some big plans this quarter, and I think that the important bits are going to work out on schedule this time, given that there is still two months left and quite a bit of the work has been started. Around Quarter 3 I’ll probably be working out the rough edges and starting on the public API.

While I’m working on the new crawler bot I will also need to rework some of the regular expressions and this may cause some of the data parsing to break for some networks. If, after the migration to the new crawler, you notice that data does not look correct please notify me via the support forum so that I can get it fixed.

Short Term Goals

These are goals that are planned to be wrapped up by the end of Q2:2015 (Q2=April 01-June 30): (Items that are crossed out have been completed)

  1. Network Administration Panel
    1. Manage users who have access to administrate networks
    2. Grant users the ability to “claim” networks securely by a process which connects to the network and verifies IRCOp access.
  2. Network Listings
    1. Featured Networks
    2. Categorize & Tag Networks
    3. Browse or Search by categories & tags
    4. Identify which listings are SFW and which ones are NSFW
    5. Split network page to pages?
  3. Improved Scoring System
    1. Removing cap on maximum score
    2. Less data driven, more people driven
  4. Crawler 2.x
    1. Crawler threads out to crawl up to 100 networks at the same time.
      1. Crawling process is expected to be under 15 minutes, where current is about 45-50 minutes.
      2. In the future crawler may attempt to stay connected to the network to avoid flooding connections.
    2. The crawler is being entirely re-written in PERL to interact with a new web API for crawling.
      1. Between the web API and threading multiple servers could be used in the future to index all networks within as little as 2 minutes.
    3. Crawler will daemonize
      1. Crawler will always check if networks need to be indexed and stay on top of the task instead of checking every 30 minutes.
      2. In the future crawler bots could try to always remain connected to avoid flooding connections.
    4. During testing period you may see a test crawl bot 2.x connecting from [email protected]esis.alpha.xnite.net, feel free to add an exception or ban the testing host name including the ident.

Long Term Goals

These are the current long term goals that are planned for completion, hopefully, by the end of 2015.

  1. Administration & Moderation Panels (Q2:2015 – Q3:2015)
    1. Allow volunteer staff to help trigger network verification process and provide the tools necessary for them to help get support requests moved along quicker.
      1. Support requests can be done quicker.
      2. Gives members an opportunity to get their foot in the door involving themselves in the process
    2. Need to work out volunteer agreement before I can start accepting volunteers.
  2. Channel Listings
    1. Channel Pages w/ Graphs for channels where there is an established maintainer and contact information is provided.
    2. Featured Channels
    3. Categorize & Tag Channels
    4. Identify which listings are SFW and which ones are NSFW
  3. Public API (Q3:2015?)
    1. View Network Information
    2. View Channel Information
    3. View Public user profile information
  4. Improved crawl bots – (Imported from unfinished tasks in previous quarter, please see Crawler 2.x in short term goals)
    1. Identify Janus linked networks where possible & automatically add Janus flag to network record

xnite

I am the founder, and lead developer, of IRC-Source. I started IRC-Source in May of 2014 because, although I don't use it as much any more, I still have a deep interest in IRC. I also carry a deep interest in web development, and wanted to write something that would be challenging, and allow me to bring my passions together in a way that would be interesting for others. I run a network called BuddyIM (irc.buddy.im), it's pretty cool, you should chill there.

More Posts - Website - Twitter - LinkedIn - Reddit