I achieved major success for my sentiment analysis and web scraping tool, as I developed two pretty cool enhancements. The first success is detecting sarcasm, which is important when you want accurate analysis results. I have been testing this for several weeks now, and it’s working extremely accurately at about an 80% to 93% accuracy rate, which far exceeds my original expectations.
My second success is that I have been able to detect nonsensical text and remove it from the analysis. This was no small task, but with some serious creative thinking, I have now achieved solid progress that results in significantly improved data that is absent of garble and meaningless text.
This functionality requires intensive processor power adding time to the analysis process, but the cleaner data makes for a far more accurate result.
As a result, I now have a pretty impressive web analysis tool that harvests well over a thousand unique news stories from all over the world, then performs a deep sentiment analysis that I would put up against any of the online sentiment tools out there.