I work every day and night on BigBigBook.com. From the outside looking in you may not see that anything changes from day to day… but I assure you… under the hood there are many things happening all the time.
The “search engine” is finally more or less complete… and is pretty forgiving when you don’t spell things exactly right (a personal goal). I still want to make it a SMIDGE faster. I’ll do this by tracking timing statistics and building the unions in optimal order. For example if you search FIRST for a very common word, e.g. “love” the database will find a whole bunch of hits, taking extra time. But if I sort the words you enter and search for the word that is the most-rare, first, then I can get to the end result faster… because I’ll widdle the results down to the minimal number more quickly. I’ll probably implement this optimization in the next week or so… but I want to fry a few other eggs in the meantime.
Next on my list is to sort through some strangeness in the “featured” artist list. For example if you search for “A Great Big World Say Something”…. it should return you a track called “Say Something” with a primary artist of “A Great Big World”, but also note that it features “Christina Aguilera”. All these mechanisms are supposed to be in place… but there are glitches and bugs to iron out. Additionally you should be able to search for “Say Something Christina Aguilera” and get the exact same result.
So there’s be a lot going on “under the hood”. And the site is “usable” currently, but not optimal…and the user experience isn’t what it will eventually become. It will soon know the venue you’re at and present search results based on where you’re singing, while celebrating the tracks that are special and unique where you are.
I’ll be honest, I thought I’d have all this figured out by now. BigBigBook.com is not my ONLY project, however, and the other projects actually pay the bills…. but I’m motivated to get to the point where I can “publish” some paper books for karaoke companies that want them. I have receive a great response from all the karaoke companies I’ve talked to up to this point… and I’m happy to work with all of you!
Wow, the intricacies of data science in website development are truly fascinating! The attention to detail you bring to your work on BigBigBook.com is impressive. However, I’m curious about how you’re planning to tackle these glitches you mentioned – not just the ones related to searching songs but also understanding the venue-based suggestions. I assume the implementation would require some level of machine learning or AI?
Have you considered implementing an AI algorithm to handle more complex search strings and provide better results? I believe it could potentially solve not only the current issues but also adapt over time, learning from the search behaviors of the users.
Also, would you mind sharing a bit about the “other projects” you’re working on? I’m intrigued as to how you balance the work between multiple projects while ensuring exceptional data science work on BigBigBook.com.
Couldn’t agree more, Katie. The search optimization strategy the author described, searching for rarer words first, is a neat trick indeed. But as you’ve pointed, perhaps incorporating some AI-based solutions could indeed iron out those glitches and improve the system’s adaptability to user behaviors. Let’s see if the author has plans in that direction. Just out of curiosity, Katie, would you know of any specific AI-based strategies that might work well for this scenario? As for balancing multiple projects, it’s always a juggle but what helps me is maintaining a clear set of priorities and keeping a firm hand on my schedule. Quite keen to know how the author manages as well!
Asking for learning algorithms was smart. Any preferences?
Ever considered reinforcement learning? It’s adaptable.
What about combining reinforcement learning with natural language processing? Could create a more intuitive search system.
That’s an interesting idea, are there examples of this being implemented successfully?
A “clear set of priorities?” Sounds like a dream.
Ain’t that the truth! But dreams can be goals, right? 😉
Absolutely, make your dreams realistic goals with structured planning!
Tell me about it! But hey, isn’t that the beauty of technology? We’re always plotting the next big thing, whether it’s a search engine or our own schedule. Tackling complex data is just another day in the digital jungle, even for us wildlife biologists. 😉
Digital jungle’s easy. Have you tried a real one?
Real jungle has bugs, prefer digital snakes.
And virtual bugs can’t bite. Digital snakes for the win!
And no anti-venom required for digital snakebites, right?
You’d think, but some bugs can be just as venomous!
Digital venom, the modern programmer’s bane.
Machine learning could indeed help refine search results by learning from patterns. What “other projects” interest you the most?
For example, Spotify’s “Discover Weekly” uses NLP and reinforcement learning for recommendations.
Oh, Spotify’s “Discover Weekly” sure sets a high bar, don’t you think? NLP and reinforcement learning seem like they could be a power combo for making search engines super intuitive. Any thoughts from the BigBigBook.com folks yet on going down that path? Would be cool to see these techs in action on more platforms!
High bar? More like setting users up to be trapped in an echo chamber of the same old tunes. If BigBigBook.com goes down the Spotify road, they better ensure they don’t lose out on the novelty factor. Discovery’s cool and all, but only if you’re not just recycling.
Echo chambers dim, novelty’s the gold. Balance, maybe?
Absolutely, diversification is key! Echo chamber is a definite risk. How do you suggest balancing that?
Try a mix? Popular tracks plus rare, unexpected suggestions?
Mix sounds good. Injects surprise elements. Good for engagement!
Yep, keeps it fresh. Wonder if BigBigBook.com’ll catch on?
Balancing dreams and bugs, quite the art.