Is it possible to tune the wordcloud generator to track phrases instead of individual words? I think we'd all be shocked to hear not only how often Phil says "um" but how often he uses filler phrases like "the bottom line is" and "it is what it is"
It is, however the number of apps and shit that can do this is fairly limited. I guess I'd prefer to do it in a way that's actually "useful" and not just easy.
What you'd ideally need to do is:
- Input every single prestream into a database separately by date
- Generate a corpus (for the non-autsitic version of "corpus" you'd just say vocabulary but this shit is even more autistic than the DSP subforum so they have special words for everything)
- Using an algorithm, analyse the concurrence or "collocation" of words.
Now you're probably saying "nigga what the fuck is concurrence or collocation".
That's where the problem starts. Concurrence is what the word autists use to mean phrases, specifically meaning "non-specific phrases". So instead of looking for exact word patterns you would be looking for words which appear alongside other similar words.
Then when you look further you go from autism to turbo autism:
http://www.ai.mit.edu/courses/6.891-nlp/aaai97.pdf
http://wing.comp.nus.edu.sg/~antho/D/D10/D10-1064.pdf
https://pdfs.semanticscholar.org/2ae3/c9a28bdeafdccc67b1648a76e7194d82cf40.pdf
So there are actually programs that can do this kind of analysis, but I haven't found any that work in the way you might expect them to when it comes to a daily prestream. Besides that, the programs that do this kind of stuff are for use in libraries and research for text that contains punctuation and is "accurate". Prestream transcriptions are not perfectly accurate.
If DSP just says a certain phrase 5-6 times during a single prestream then that's boring. If DSP says some very specific shit about his mortgage 3 times a month and then suddenly starts saying it 16 times a day, that's a lot more interesting. So, if I just dump out the phrases without their usage over time its pretty much irrelevant.
And I really could just dump out all the phrases he's ever used on prestreams except DSP's entire prestream text adds up to 16.8 MB in size.
16.8 MB of text is a large amount of text. Its a fucking huge amount of text.
The King James Bible is 4.8 MB of text. DSP speaks more than Jesus ever did because he is our lord and saviour so DSP's prestreams for just 3 years are
equivalent to 3.5 bibles worth of text.
No website on earth that offers "simple" shit that will do this analysis will let you put more than 0.5-1MB of text. I did upload all the text to some open source word analysis project thing, but the developers
must've gotten really mad about a bunch of kiwifarms autists making fancy graphs and causing their servers to catch fire with the holy word of DSP that they IP banned me.
So I tried Windows programs that can do this shit, but
they all crashed because of how magnificent and large DSP's holy grail is. The only ones I didn't try are like actual research programs that cost hundreds of dollars.
Then it kinda changes from being about "concurrence" to "collocation"
There are command line programs, however installing and using that shit is super fucking complicated and I haven't gotten around to it just yet.
By complicated I mean this level of complicated:
http://www.nltk.org/howto/collocations.html
However, getting that shit to work takes time and autism but also still doesn't solve providing the phrases with actual context: how many times have they been used and at what times have they been used? Because this shit was designed for word autists and books and not analyzing DSP's prestreams.
I do think it would be super interesting to make a project like this, not just for DSP but also any other lolcows or for whatever use it might have for someone else but I'm not sure I'm even autistic enough to accomplish it. Although if I continue to be the supervisor on the DSP subforum for a serious amount of time it is possible I will achieve an autism singularity and become one with sonichu.
But reaching that autism level is
dangerous territory. I'd probably start cosplaying and interacting with furries and instead turn my attention to SJW issues and blue arms of famous hedgehogs.