Author: xnite

I am the founder, and lead developer, of IRC-Source. I started IRC-Source in May of 2014 because, although I don't use it as much any more, I still have a deep interest in IRC. I also carry a deep interest in web development, and wanted to write something that would be challenging, and allow me to bring my passions together in a way that would be interesting for others. I run a network called BuddyIM (irc.buddy.im), it's pretty cool, you should chill there.

Postmortem: Inconsistent Data

Over the past week our data has been all over the place on the global graph.
This was due to a migration process in which we put a VPS up for a new database server, and it simply could not handle the load.
We have since then swapped out the database server with a new dedicated server which costs less, and will handle the load more efficiently (yes I said costs less, it’s because we’re awesome like that). Since swapping out the server data has started to look less erratic, and the infrastructure as a whole is a lot more stable.

<3 – The IRC Source Team

The blue line represents the average CPU usage of the infrastructure as a whole, while the red line represents the (old) database server
The blue line represents the average CPU usage of the infrastructure as a whole, while the red line represents the (old) database server

Continue reading “Postmortem: Inconsistent Data”

Expected Downtime

We are expecting downtime for all web, and crawl services sometime between the window of 6AM – 2PM UTC-0 on Friday, August 18th 2017.
During this downtime we will be performing maintenance on our database and web infrastructure which is expected to last at least 1 hour while we get everything migrated and reconfigured.

All systems go – July 31st

As of July 31st we will be putting the new crawl servers into effect. The new servers will have the hostnames yin.ircs.me and yang.ircs.me.
The new addresses for the crawl servers are:

yin.ircs.me
49.56.142.157
2607:5300:201:3100::13dd

yang.ircs.me
149.56.142.163
2607:5300:201:3100::299d

Please update the exempts in your IRCd configurations to include these IP addresses before July 31st to avoid any disruption in crawling your network.

New Features

  1. The new crawler contains a master-slave system for added redundancy in crawls.
  2. The new crawler grabs up MOTD for each individual server, and will be accompanied by a website update.
  3. The new crawler allows staff more control over the index and status of a network’s index.
  4. The new crawler is able to handle networks which previously the crawler struggled with.
  5. The new crawler is much more fault tolerant.
  6. The new crawler is able to crawl at lightning speeds!

 

IRC-Source Crawler 3.x Progress

IRC-Source Crawler 3.x is currently successfully updating InspIRCd based networks. We will be testing the crawler on various IRCd’s once we have tested the stability of the new crawler. The crawler is designed to scale up quickly with it’s Master/Slave system, so we are planning on using 2 crawl servers at first to see how things go. The new crawler, and scaling, is necessary because the current crawler uses all CPU and memory when crawling which could lead to some of the errors that we have been having with indexing certain networks.

Continue reading “IRC-Source Crawler 3.x Progress”

Happy Birthday to IRC-Source

The first ever recorded line of history in the IRC-Source database is 2014-05-27 23:57:48 UTC-6. We consider this date to be the birthday of IRC-Source. In recognition of IRC-Source turning 3, here is a brief history of IRC-Source, and some of our goals.

In 2016 2 new members were added to the IRC-Source team, James Lu and Ken Spencer.

Continue reading “Happy Birthday to IRC-Source”

Missing Crawler Data Post Mortem

Incident Details

Last night we tried to only crawl networks once per hour and wound up with missing crawler data. Usually we crawl at the beginning of the hour, then again 30 minutes later to crawl missed networks.
Due to a slight time difference between the crawl server and the database server some networks were being indexed at the end of the hour.
We crawl networks backwards from the last time each time. Meaning if Freenode was first one hour, then it will be last the next, so on and so forth. This caused data on large networks to be crawled at the end of the hour every other hour.

Conclusion / Fix

We have implemented an ntp client daemon on both servers, but just in case of future time desyncs we start indexing 1 minute after the hour. We will be monitoring the situation, and if this becomes a problem we will set it back.

May Updates

Community Updates

A fix for the forums has been implemented which now allows for any member to post to the forums.
Please remember that forums are still in alpha stages, and many features may be missing or incomplete.

Forums

The forums activity has been pretty low recently, we could really use your help to get things going even if you just come on to post an introduction for yourself and/or your network. Every last post helps get our community moving along.

Graphs

New graphs have been added, and some changed to display information a little bit better.
We now have 7 day history, last 30 days, and more. You will also notice that the home page displays all statistics over the past 24 hours.
You may embed these ne graphs into your website to show off your statistics to your visitors.

Development Updates

 Bug Fixes

  • Forums have been fixed so members can now create posts and comment.
  • Login now allows email OR username, unless migrating your account from our old WordPress website.
  • CAPTCHA issues have been resolved by creating a custom CAPTCHA solution
  • Various other bug fixes and text updates

Other Improvements

  • Search pages start off clean
  • Faster sessiona handling + persistent sessions across each web server.
  • IRCCloud links have been added to channel and network pages
  • Patrons get an ad-free role allowing them to bypass advertisements
  • Changed markup to allow linking usernames by using /u/username for example.
  • Improved website by loading form responses with jquery instead of using nasty redirects.
  • Various SEO improvements to increase search visability.
  • Various performance optimizations.

Into the future

We are currently working on an API for IRC-Source, and will release more details once it’s finished.
The forums and moderation panels still need some work, and I’ve mostly been working on the backend (Admin Panel) lately. Unfortunately our advertiser (Google Adsense) has laid out some ground rules for us to abide by in order to keep ads up, which currently fund the site. If you have even $1 a month to spare, please become a patron.

Goals made, promises kept

After making our $20 goal we made the promise to split the crawler up onto multiple servers to reduce crawl load. We also made the promise to enable IPv6 support on the crawl bot, which we have done. We also got a new crawl server, check exempts for up to date exempt blocks.