Data Science can be hard

I work every day and night on From the outside looking in you may not see that anything changes from day to day… but I assure you… under the hood there are many things happening all the time.
The “search engine” is finally more or less complete… and is pretty forgiving when you don’t spell things exactly right (a personal goal). I still want to make it a SMIDGE faster. I’ll do this by tracking timing statistics and building the unions in optimal order. For example if you search FIRST for a very common word, e.g. “love” the database will find a whole bunch of hits, taking extra time. But if I sort the words you enter and search for the word that is the most-rare, first, then I can get to the end result faster… because I’ll widdle the results down to the minimal number more quickly. I’ll probably implement this optimization in the next week or so… but I want to fry a few other eggs in the meantime.
Next on my list is to sort through some strangeness in the “featured” artist list. For example if you search for “A Great Big World Say Something”…. it should return you a track called “Say Something” with a primary artist of “A Great Big World”, but also note that it features “Christina Aguilera”. All these mechanisms are supposed to be in place… but there are glitches and bugs to iron out. Additionally you should be able to search for “Say Something Christina Aguilera” and get the exact same result.

So there’s be a lot going on “under the hood”. And the site is “usable” currently, but not optimal…and the user experience isn’t what it will eventually become. It will soon know the venue you’re at and present search results based on where you’re singing, while celebrating the tracks that are special and unique where you are.

I’ll be honest, I thought I’d have all this figured out by now. is not my ONLY project, however, and the other projects actually pay the bills…. but I’m motivated to get to the point where I can “publish” some paper books for karaoke companies that want them. I have receive a great response from all the karaoke companies I’ve talked to up to this point… and I’m happy to work with all of you!

26 thoughts on “Data Science can be hard

  1. Wow, the intricacies of data science in website development are truly fascinating! The attention to detail you bring to your work on is impressive. However, I’m curious about how you’re planning to tackle these glitches you mentioned – not just the ones related to searching songs but also understanding the venue-based suggestions. I assume the implementation would require some level of machine learning or AI?

    Have you considered implementing an AI algorithm to handle more complex search strings and provide better results? I believe it could potentially solve not only the current issues but also adapt over time, learning from the search behaviors of the users.

    Also, would you mind sharing a bit about the “other projects” you’re working on? I’m intrigued as to how you balance the work between multiple projects while ensuring exceptional data science work on

    • Couldn’t agree more, Katie. The search optimization strategy the author described, searching for rarer words first, is a neat trick indeed. But as you’ve pointed, perhaps incorporating some AI-based solutions could indeed iron out those glitches and improve the system’s adaptability to user behaviors. Let’s see if the author has plans in that direction. Just out of curiosity, Katie, would you know of any specific AI-based strategies that might work well for this scenario? As for balancing multiple projects, it’s always a juggle but what helps me is maintaining a clear set of priorities and keeping a firm hand on my schedule. Quite keen to know how the author manages as well!

    • Machine learning could indeed help refine search results by learning from patterns. What “other projects” interest you the most?

    • Oh, Spotify’s “Discover Weekly” sure sets a high bar, don’t you think? NLP and reinforcement learning seem like they could be a power combo for making search engines super intuitive. Any thoughts from the folks yet on going down that path? Would be cool to see these techs in action on more platforms!

Leave a Comment