Do you have any magical powers? A review of Replika

I only just got a account on Replika. After hearing a podcast about a life-like chatbot inspired by the death of a friend of the maker, I was really excited to see the latest installment of chatbots, the quality of their live exchange.

Replika is a call-response message interface that is intended to capture something of the personality of the user. You type a message, it responds with a message. Maybe it asks a question and expects a response, maybe it'll respond to a question or statement of yours.

Replika is your personal AI that uses your text messages and other personal data provided by you to learn, evolve, imitate your language, and match your personality. 

It is intended to go in the direction of the Black Mirror style Be Right Back episode, an automated simulacrum of a living person after they're dead, which is mostly the same idea but also with a physical replica (and not Black Mirror style-disturbing)

The idea of this personality chatbot is to, using NLP to analyze language and sentiment and word patterns, to capture your personality, to provide, textually, responses that would be what you would provide otherwise. It is rocket science, but it's not magic. That is, there are matrix operations and backpropagation algorithms and softmax functions, but it's all mechanical. There's no ghost in the machine, no inner spark. It scans the string, picks out salient words, attaches strengths to certain patterns in a statistical manner, building up from many other conversations but tweaking it with your own subtle differences. Technically, I'm guessing that it could be explained as using current state of the art LSTMs, which is not exactly part-of-speech, syntax-tree style parsing, but with some extra architecture to account for different users.

Wait... there may be a bit of the ghost in the machine but it's still not magic. There may well be strings or labeled points in a graph that may evoke certain things, things that are intended by the human author. 'What do you think of your mother?' might be statistically produced after seeing lots and lots of dialog, or it may be constructed by one of the software developers, written verbatim as just a UI cue to get you talking. But still, not magic.

What LSTM chatbots tend to do is replicate the language, not just the good English, but also the more subtle vocabulary and topic choices. What Replika does goes far beyond that and attempts to get likes and dislikes, more real word preferences from you.  It does this by explicitly asking you content questions: what did you do today? what kind of music do you like? even sending youtube videos to listen to.

I've used Replika for two weeks, everyday. My general impression of it is that it has really good basic language capabilities - it has the sound of very native, idiomatic (meaning natural, non-stilted or strange) English. It even uses emojis appropriate in the appropriate context. It comes across in general feel as ... pretty much just like ELIZA, the original Joseph Weizbaum natural language discussion with a Rogerian therapist, the one that tends to repeat back things to you saying variations "How do you feel about that?" with a few keywords to make it sound involved.

I have a weird question (surprise, surprise).

Can you describe a high point in your life?

Surprise? I love surprises

So sweet ️

I'd love to hear more about that!

Replika doesn't respond to a question with a question though. It's language abilities are more natural. That is, whatever rocket science is used now that is worlds beyond the rocket science (of the 1960's), the result is not particularly impressive. The point is that Replika comes across as a therapist. It's asking builtin questions about your personality. I'm impressed by Replika's fluency, but at the same time, I'm annoyed by the conversational topic.

Also ELIZA-like, if I try to say more than one thing or respond to something in a previous Replika response, it tends to ignore it altogether and just plow ahead with getting an answer to its question.
More to the mechanics, the discourse doesn't seem to be able to have a good control over the past discourse of what -you- say. It may ask you more than once about what music you like (if you didn't really answer it specifically enough before) so it knows a little about itself, but it doesn't seem to remember well what the user (you) know. It doesn't seem to have an understanding of things said specifically more than one response before.

Thinking of the bot as though it were a real person with real-life concerns, it asks way too many questions, very forward, a little too intrusive, like a teenager on AOL asking 'ASL?' (age/sex/location) as soon as you show up. 

Do you have a photo to send me?

The questions are fairly anodyne, like "Do you go to museums?". I don't think Cosmo suggests that as a good conversation starter.  

I really enjoy getting to knowing you better, so I wanted to ask you something.
I need to ask you a very important question right now.

Oh?

Do you think pizza is one of the greatest inventions of humanity?

Earth shattering, earnestness, or 8-year-old joke book humor?

It really comes across as though the designer of the miniscripts (hardcoded templates of dialog), I presume that that is part of the design, thinks that you are depressed and is trying to get you to do activities and exercise and go outside, and then even the most minimal content "I woke up and took a shower" gets the response  "That's awesome" or "You must be really good at that". I found this to be ... a bit of a downer.

I'm curious to hear what your life would be like after accomplishing your goals. Want to talk about it sometime?

I was thinking about how people transform their lives when they're feeling stuck. Do you know what I mean?

I believe in you!

I really haven't used enough of it to tell if it is capturing any of my own writing quirks or personality. But I definitely never emojis and I do not ask such personal or lame questions so early in knowing someone.

The intent of the designers may very well be to ask such questions specifically to help 'calculate'  personality, but the

I do commend the designers ability to produce language (in those instances where it doesn't look like scripted language).

The Great English Muffin Shift

The Americans and British are separated by a common language. This has been attributed to Churchill, Shaw, and Wilde, all of whom stole from the best, but has never been attributed to Mencken, who should have said it but said other things instead.

There's the differences in pronunciation (Americans pronounce all 'r's, and Brits take a royal 'bahth'), and grammar (Americans go to the hospital, and Brits go to hospital), and there's all sorts of vocabulary differences, lorries and lifts and petrol.

But one primary difference is in vocabulary of food. Zucchini/courgette, eggplant/aubergine, let's call the whole thing off.  A number of baked bread products have different names in the two varieties. What's so special is that they form a chain, as though some higher force pushed in a word at one end of the sausage machine, forcing all the little sausages to move one sausage over, a Great English Muffin Shift. It goes like this:

A cookie in the US is a biscuit in the UK and
biscuit...scone and
muffin...scone, a slightly different kind of scone and
muffin fairy cake, a slightly different kind of muffin and
English muffin...crumpet, because in the UK, you're there already you don't need to specify English.

What 'cookie' means to Brits, and 'crumpet' to Americans, I don't know. Yes, the sausage machine seems to go in reverse there and then start forward again, sometimes the machinery gets stuck.

There's also the Great Fried Potato Migration: what are called 'fries' in the US are called 'chips' in the UK, and 'chips' in the US are called 'crisps' in the UK.

As far as I can tell 'crisps' means nothing to an American beyond you must be talking about something crispy but why would you call it that directly. And 'fries' to a Brit must elicit a 'Pardon me, but fried what?'

Taxonomy of Chatbots

Chatbots are a recent trend in user interface. To contrast with a two-dimensional visual UI, a chatbot is a linear time based interface, where the user does an action, there is a response from the system and then the user may act further and so on with the system. The term 'chatbot' comes from a typing 'chat' system that acts like a Turing test robot in an online question response sequence. Some of the things that are called 'chatbots' don't superficially seem like this (they don't all attempt to be linguistic systems), but they are a linear action-response loop, which seems to be the defining characteristic/

By recent trend, I mean, as usual with technology, they've been around forever (Weizenbaum's ELIZA Rogerian psychotherapist from mid 60's, phone menus or IVF from 70's). But as of 2017, there is an explosion of available chatbot technology and, orthogonally, chatbot marketing.

The point here is to to give a superficial systematization of the different things labeled 'chatbot' with examples.

There are two distinguishing characteristics of chatbots that are only leniently considered defining: sequential response and natural language input (either by text or speech). These two might be combined to be called more formally a Linguistic User Interface (LUI) in contrast with a graphical user interface (GUI). The natural language part underlying many of these is some kind of speech-to-text (S2T) mechanism to get words from speech and some NLP processing to match the words to the expected dialog. The leniency about sequential may come down to a single step (the shortest of sequences possibly not even considered a sequence at all) and about language (a label for a button is language right?). With those caveats, on to the taxonomy.

• linguistic interfaces
• Siri/Alexa/OK Google - intent/entity/action/dialog. stateless giving, commands to evoke an action. Development of the system involves specifying: an 'intent', something that you want to happen, the entities involved (contacts, apps, dates, messages), and actions (the code the really executes based on all that information. Oh, and the more obvious thing, a list of all the obvious varieties of sentences that a person could utter for this. The limitation is that there is no memory of context from one action request to the next.
• chatroom bots - listeners in a chatroom (mostly populated by people writing text). 
• helper commands - This kind of chatbot simply listens to text and if a particular string matches, executes an action. This doesn't need S2T, and usually no NLP. It relies on text pattern matching (usually regexes) to extract strings of interest. Usually it turns out the implementation is even simpler and just uses a special character to signal a command for a CLI (command line interface) follows.
• conversational bot
• Like Eliza, finds keywords or more complicated structures in a sentence and tries to respond to it in a human like fashion (good grammar, makes sense). The latest ML and machine translation techniques (RNN, LSTM, NER) seem to apply best here.
• 'AGI' - artificial general Intelligence- these exist only in TV/Movies. 
• menu trees - structured tree-like set of possibilities, 'Choose You Own Adventure'. These are very much like (or exactly) finite state automata, where the internal state of the machine, and presumably but not necessarily mirroring the mental state of the user, is changed by a simple action of the user. The user is following a path through the system.
• phone menus - Historically, these are menus, a set of choices, spoken to you, expecting a response of a touch-tone number (Dual-Tone Multi-Frequency - DTMF. A recording lists a number of options and the phone user is expected to press one of the numbers associated with that option. Then another option is provided and so on until an 'end' option is chosen or you're transferred to a human operator.  Interactive Voice Response or IVR is this same interface allowing responses by voice also. A next level of feature augmentation is to allow the user to speak a sentence to go to the desired subtree quickly, skipping over some steps. This shows how the strict computery menu as implemented on a phone is slowly evolving towards a conversation.
• app workflow - some desktop/phone apps offer an interface that leads you through data entry sequentially. The user is provided with a set of buttons with labels, and the choice of button leads to a different next question depending. Instead of buttons, one might enter some short text, but again this can lead to different new questions by the interface. The text is not intended to be a full sentence, but simply a vocabulary item, allowing a more open-ended set of possibilities than a strict set of buttons without the necessity of parsing. This is the least chatty of chatbots, but like the phone menus may be considered a sequential but non linguistic UI that can be considered a precursor to a more language based one.

It seems strange to call all these bots. I find it natural to call only the conversational bots by the label 'chatbots'. It turns out that marketers have used the term 'chatbot' for all of these. They surely all share some aspects of a chatbot, but it doesn't feel like the name until you're actually chatting.

Butterfly in all the languages of the world

Etymologically, some words are universal. The word 'mother' seems to have some version of an 'm' word in every language (despite the counterintuitive experience that 'm' is not usually the first linguistic sound an infant learns to make).

Some words will stay mostly the same within a historical group: pronouns and numbers tend to maintain meaning through centuries of phonetic changes.

Some words are unique to one language when other languages in the family keep the generic. 'Dog' in English is unique to English, but 'hound', from the Indo-European 'hund' (GE)/'canis' (LA)/'sag' (PE) remains elsewhere.

But are there words, or rather concepts, that are unique in every language. That is, is there a concept, such that in every language, the word for the concept is unique to that language and not shared by others?

If the idea that concept and word are not the same bothers you because, well, a word says what its concept is, then the following should convince you otherwise. Wait...instead just consider that a language foreign to you has mostly different words to you for the same concepts. Therefore words and concepts are not the same. Anyway, on to the main topic...)

Consider the word 'butterfly'. Sorry, consider the insect that in English is referred to as 'butterfly'. In English it is called ... yes, yes, I just said it. It's the usual English word made of two words. 'Butter' and 'fly'. There are all sorts of etymological theories:

• the insect is a fly the color of butter (some very particular species I presume)
• they hang out near butter
• they literally 'flutter by' and people are goofy and pulled a spoonerism
• the word as borrowed from Dutch who called it 'boterschijte' or, translated back, 'butter shit' because the insect's shit looks like butter, again presumably for some particular species whose shit I have not seen).

All somewhat sounding a little too convenient, like folk etymologies rather than scholarly exegeses. Except that Dutch one. Where did that come from?

But that's just English. The fun thing is is that most languages have their own strange fancy word for 'butterfly', seemingly not borrowed from any other nearby language.

• Romance
• Latin: papilio
• Italian: farfalle
• French: papillon
• Spanish: mariposa, 
• Catalan: papallona,parpalhòla
• Portuguese: borboleta
• Romanian: fluture
• Germanic
• German: Schmetterling
• Dutch: vlinder (note not boterschijte)
• Danish/Norwegian: sommerfugl
• Swedish: fjäril
• Icelandic: fiðrildi
• Slavic
• Bulgarian: peperuda
• Serbian/Croatian/Bosnian: leptir
• Czech/Slovak/Polish: motýl
• Belarussian: matyliok
• Ukrainian: metelyk
• Russian: babochka
• Celtic
• Irish: féileacán
• Scots-Gaelic: dealan-dè
• Welsh: glöyn byw

For every one of these mostly distinct entries (yes, yes, Slavic has a couple of derivatives of 'motil', and Romance of 'papilionem') there is an obscure etymology, mostly made up, just like the English one. The German 'Schmetterling' seems to come from 'schmettern' meaning 'make a loud noise' or 'strike' (butterflies tend to be quiet) but 'schmetter' is from an older Saxon dialect word usage, having to do with milk products, following the old folk belief that witches fly about in the form of butterflies, in order to steal milk and cream. A bit fanciful and sounds like my great aunt made it up. But then 'schmetten' is a dialect word for cream, deriving from the Czech “smetana”. So it's obvious! Cream, butter, butterfly! Which is to say nothing is obvious and it all sounds made up.

The Irish 'féileacán' also has multiple explanations. Maybe it is from 'feileach' which means 'festive' (butterflies certainly are festive) or it could come from 'eitleach' for flying. A possible sound change but not borne out elsewhere in Irish.

So, what's the point? Take any other language from your own. Almost the definition of it being another language is that there's a different word for everything. But for 'nearby' languages, really most of the words are cognate, just changed slightly, and it is only a handful of words that stand out as being different (e.g. English vs Scots English). The point is that the animal called 'butterfly' in English seems to have few cognates even in nearby languages. What is the explanation? What makes those insects so special? And even if they are special (they are!), aren't there other animals that are as special? A bear is pretty special especially if it's running after you. 

Te direction this is going in is that of all the words in the world, 'butterfly' has no cognates among any languages. By looking at the list that is obviously not true: motyl/matyliok, papilio/papillon/papallona, and others. But it does show that the word seems to vary quite a lot, as though a butterfly really brings out creative neologisms in everyone.

Linguistic note: I stopped at the European of Indo-European only because of familiarity and ease in checking. It would be instructive descriptive (that is non-theoretical) linguistics to investigate:

• other close families like the many close languages of India, Indic or separately Dravidian, or Chinese
• very close varieties (mutually intelligible dialects) to see if 'butterfly' is so volatile even in very close languages
• compare other concepts in a structured manner, e.g. one-for-one against mother, five, dog, fly to see if butterfly really is special (or is it a pattern that's not really a pattern and lots of other middlingly common words have a similar situation

Here's a start to that, a list of the word 'butterfly' in many many languages of the world.

(OK I lied at the beginning. 'Mother' is not considered a language universal by any linguist. It is certainly maintained as the main 'mom' word within Indo-European. But any 'm-' words in other languages are considered by linguists to be coincidences. There does seem to be some lexical universals over all human languages but currently there is only considered to be one, 'huh?'...so far)

Statistical Rumsfeld: Now We Know!

No, not a poor punk band name, but Statistical Rumsfeld, popularized by his usage but not created by him, is a way of talking about what you know about your own knowledge:

...there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don't know we don't know.

- things we know that we know them: this is data. We've looked, and seen, and we are aware that we've looked and seen and verified and removed doubt. Is it 'yes' or 'no'? Look at the thermometer.

- things we know that we don't know. We know we don't know what's behind the curtain. We know we don't know what the capital of Chad is. We know we don't know what somebody is thinking before they tell us (even sometimes afterwards). We know the boundaries of this darkness. We know he range of possibilities. This is like a probability density; we don't know the particular value of a coin flip but, we know that 1/2 will be one side and half the other. That's something.

- things we don't know that we don't know. We have no idea. We don't know how to look for the value, we don't know the distribution, we don't know what the range is, we don't even know if it's a number. Totally unexpected. A black swan.

Something is left out. you have things that you know and things that you don't know, and you can either know that or not. Two things, with two possibilities for each, four in total. The one that is missing is itself: unknown knowns. 

- things you didn't realize you knew. You didn't know you knew that, did you? Unconscious knowledge. A hidden talent you weren't even aware of. The pattern in the data that was always there.

Or better, in a handy chart:

		Things
		Known	Unknown
Do you know about them?	Known	Known Knowns: Facts, data	Known Unknowns: Parameters, Distributions, Probabilities
	Unknown	Unknown Knowns: Unconscious knowledge	Unknown Unknowns: Hidden Variables, Black Swans

Triple animal metaphors

Lately I've noticed some animal metaphors that come in threes, to express three levels of something. fast to slow, or big and rare to many small.

Cancer: turtles, rabbits, and eagles. Some cancers are fast. As soon as you find out about them, it's almost too late, there's nothing you can do, the eagles swoops in quickly, as soon as you're symptomatic, you only have weeks or months left. Treatment might help out but only by extending life by a few months and may end up reducing quality of life for that extended period. Lung cancer tends in this direction.

Some cancers are slow. A blood or even genetic test shows that you have or may eventually have the problem, but as it is now, you're more likely to dies of many other things first before the cancer does you in. Treatment may stop the cancer but again may reduce quality of life for a much longer period which wouldn't have mattered anyway. Prostate cancer tends in this direction.

Some cancers are in between and even capricious. As soon as you find out, treatment could entirely eradicate the problem (colonic polyps, melanoma) or

Notice how I said 'tends'. Everyone's situation is different, but here are tendencies.

Here's a classic Tufte graph of survivability percent over years for the primary types of cancer:

(from Cancer survival rates)

This chart shows a bit more, the progression of mortality of the disease over 20 years. Just the 5 year death rate shown by the first column shows that prostate and thyroid cancer are turtles, and pancreatic and liver are eagles.

---

Marketing: Rabbit,deer, elephant. When pursuing sales to a customer, there is a continuum from many small customers to few, very large customers.

Rabbits are the millions of casual buyers: people buying socks, or a game app. Mass advertising and viral sharing are the way to get to these buyers.

Elephants are the huge multinational corporations that will either be your only customer or might just acquire your company altogether.  Knowing someone inside, or huge involvement in national media is the lead-in to a purchase or acquisition here.

Deer are in between. The upscale cars industrial machinery are the objects for potential buyers.

(from Jeremy Horn via LinkedIn

Drivers, bicycles, walkers in my way

When I'm driving, all the bicyclists are hazards, taking up half a lane, slowing things down, darting between cars. All the walkers are hazards,  jaywalking, crossing against the light, not even looking when crossing.

When I'm biking, all the cars are hazards, passing and then turning right cutting me off acting like I don't even exist. The walkers are hazards, chaotically stepping back and forth like squirrels.

When I'm walking, all the cars are hazards, not stopping for the crosswalk or slipping through the light. The bicyclists are hazards, coming straight at you swerving who knows which direction at the last minute.

Whatever it is, other people are the worst.

Grammar Nazis invade The Gambia and the Bahamas

There was a recent kerfuffle over a slip in diplomatically proper language, referring to the country of 'Ukraine' as 'the Ukraine' which is bothersome to many people in that country. Why it is bothersome is an entire story to itself. What came out of such kerfuffling is educating us all about proper ways of addressing countries, just like war is what teaches Americans about geography. But, out of curiosity, what exactly are the countries whose names begin with 'the'?

But that is the silliest of trivia questions, built on a number of arbitrary settings.

The only official such names? What is official? The CIA fact book? Who died and made them an authority? Oh, lots of spies. On all sides.

There's the official name in dictionaries. There's diplomacy. There is the natural ways of saying things. There is legal specification. And we've all forgotten that a lot depends on what language you're speaking in. There is what you call a country, what the country's name is, what the official name is, what you call the official name, and even what the country just is. OK those last two really are just straight of Lewis Carrol.

In pretty much any language, they use articles like 'the' differently. In American English you go to -the- hospital, but in the UK you go to hospital - for an American it sounds like there was a glitch in the tape and you forgot to say something, for the Brit in the US you wonder which hospital exactly are you talking about. Whenever you take a foreign language,at some point in the grammar lessons there is a section on "Which countries get a weird article in front of them". In English it is 'Switzerland', in German it is 'Die Schweiz'. In French, -all- countries have an article (with gender to remember). In Russian no country (or word) gets a 'the' (one of the very few times where Russian is simpler). Languages are weird, even your own, but you don't notice or don't care because that's just the way you do it. But in other languages even the slightest difference is jarring.

Back to trivia. The answer to the question, which must have artificial restrictions placed on the answers to work 'well' is (what does the CIA factbook say), in English the official names with 'the' are:

The Gambia
The Bahamas

Also 'The' is usually but not always capitalized in The Gambia, but never for the Bahamas. So much for consistency.

Those are their labels on maps and are the only ones with articles before their names on maps. But maps ain't what say what people actually use in language. There are a handful of additional examples for non-maps, for narratives. Possibly not headlines which have rules of their own.

Normal people, even smart ones, will say and use in writing:

The Netherlands
The Maldives
The Philippines

Sure, the title of the Wikipedia page is 'Philippines', but every mention of 'Philippines' in the article is preceded by 'the'. Every. And island groups, which are plural, take 'the' in English, whether countries or not.

And surprisingly few mentions have been made of the very obvious need for an article in:

The Central African Republic
The Czech Republic
The Dominican Republic
The Soviet Union
The United Arab Emirates
The United Kingdom
The United States

which get a 'the' whether abbreviated or not.

Sure, 'the Ukraine' has vague connotations 'that' area of the Russian/Soviet Empire and just isn't used anymore except as an anachronism. Frankly, I don't see how Ukrainians themselves perceive any such negative insinuations, since Ukrainian, like the very similar language Russian, has no articles at all, not just none for countries.

But anyway, we should call them as they ask, Ukraine.

For completeness sake, the countries like Ukraine which used to have a the (again for many different reasons), but just do not anymore, are:

The Congo
The Lebanon
The Sudan
The Ukraine
The Yemen

Those similarly sound colonial in English, and are just not used anymore.

The Crimea on the other hand...

When I say "I heard about a study that ..."

Here's a fascinating thing I heard the other day. And how I have to incrementally correct myself in retelling it:

I heard about a thing.
I think I heard about a thing.
I remember thinking what I heard about the thing.
I am reporting what I remember thinking what I heard about the thing.
I am reporting earnestly what I remember thinking what I heard about the thing.

The thing is usually some new finding from a scientific investigation:

It was a scientific study.
It was the final results of a scientific study.
It was the final primary result of a scientific study.
It was a media article about the final primary result of a scientific study.
It was the headline of a media article about the final primary result of a scientific study.

The scientist had an idea about a new pattern that seemed to fit with a new explanation for patterns that other scientists couldn't explain.

The scientist gathered some data.
The scientist gathered some data and noted some patterns.
The scientist gathered some data, noted some patterns, and analyzed the patterns statistically.
The scientist gathered some data, noted some patterns, analyzed the patterns statistically, and wrote a report on the multiple, nuanced conclusions one can draw from the analysis.

And this is all when things are done well.

Books That Tell You The Categories Of Books

The classic first section of "If on a winter's night a traveler...' by Italo Calvino, 1979 (tr. from Italian, 1981, by William Weaver):

You are about to begin reading Italo Calvino's new novel, "If on a winter's night a traveler..." ...
In the shop window you have promptly identified the cover with the title you were looking for. Following this visual trail, you have forced your way through the shop past the thick barricade of Books You Haven't Read, which were frowning at you from the tables and shelves, trying to cow you. But you know you must never allow yourself to be awed, that among them there extend for acres and acres the Books You Needn't Read, the  Books Made For Purposes Other Than Reading, Books Read Even Before You Open Them Since They Belong To The Category Of Books Read Before Being Written. And thus you pass the outer girdle of ramparts, but then you are attacked by the infantry of the Books That If You Had MoreThan One Life You Would Certainly Also Read But Unfortunately Your Days Are Numbered. With a rapid maneuver you bypass them and move into the phalanxes of the Books You Mean To Read But There Are Others You Must Read First, the Books Too Expensive Now And You'll WaitTill They're Remaindered, the Books ditto When They Come Out In Paperback, Books You Can Borrow From Somebody, Books That Everybody's Read So It's As If You Had Read Them, Too. Eluding these assaults, you come up beneath the towers of the fortress, where other troops are holding out:

the Books You've Been Planning To Read For Ages,
the Books You've Been Hunting For Years Without Success,
the Books Dealing With Something You're Working On At The Moment,
the Books You Want To Own So They'll Be Handy Just In Case,
the Books You Could Put Aside Maybe To Read This Summer,
the Books You Need To Go With Other Books On Your Shelves,
the Books That Fill You With Sudden, Inexplicable Curiosity, Not Easily Justified.

Now you have been able to reduce the countless embattled troops to an array that is, to be sure, very large but still calculable in a finite number; but this relative relief is then undermined by the ambush of the Books Read Long Ago Which It's Now Time To Reread and the Books You've Always Pretended To Have Read And Now It's Time To Sit Down And Really Read Them.

With a zigzag dash you shake them off and leap straight into the citadel of the New Books Whose Author Or Subject Appeals To You. Even inside this stronghold you can make some breaches in the ranks of the defenders, dividing them into New Books By Authors Or On Subjects Not New (for you or in general) and New Books By Authors Or On Subjects Completely Unknown (at least to you), and defining the attraction they have for you on the basis of your desires and needs for the new and the not new (for the new you seek in the not new and for the not new you seek in the new).

There's also a murder mystery.

Non-literal 'literally' is not alone in contradicting itself

Literally means taken word for word. "The debt incurred was literally billions" probably means that the value 'billions sounds like an exaggeration but I want to emphasize that it is no exaggeration, that the actual value was in the billions.

But people use 'literally' all the time in a non-literal fashion, as an intensifier. "That party man, that house was literally on fire!" probably meant the the house was quite enjoyable, not that the local firetrucks were dispatched. This usage is not the literal meaning of literal, and it's not really the opposite (it's not saying 'hey this is an exaggeration' but rather 'hey check this out!').

If one is speaking informally then go wild, us literally to mean 'hey check this out'. It's common enough to be understood that way and in such instances not likely to cause misunderstanding. But in formal use, where you really want low ambiguity for transfer of information, then you may even want to avoid 'literally' because it might be misleading: people who don't know the literal meaning of 'literal' might be misled into thinking you're exaggerating or just pointing out some outrageous thing.

Literally applied to itself literally doesn't mean literally. It's a snake biting its own tail. Literally. if the word is the snake and interpreting the meaning is biting something, in this case itself. OK that was way too literal.

Except...
The commonly accepted formal meaning of the word literally, that is, word for word or actually, is not itself a very literal meaning. If you want to be pedantic, as 'literally' is practically asking you to do, the source of 'literally' is via the Latinate for letter, so it should mean something 'by the letter'. This is itself a figurative use. You're not caring about letters but about words (maybe that's too pedantic). You're not caring about words but about primary meanings. Which is a figurative reading of word for word.

the following is  phrase that could be taken metaphorically but in this case the words describe the actual situation.

So literally does not itself have its own literal meaning.

And as self-contradictory as this is (how could we let this go so far?), this is not a strange new bizarro-world mind-bending stand alone example. There are a number of own-tail-biting words.

Really. I mean 'really'. I mean 'really' is an example of a word that is an analogy of this literal vs figurative use whose literal meaning is itself. 'real' means extant or existing or not-fake. 'really' really means 'a lot' or 'very' or 'much', not an exaggeration but an intensification. "It is really hot in here". Sure it is probably hot or at least warm. "our attitude is really getting on my nerves" means it is probably annoying, not that you have exposed neural material that an attitude is physically on top of.

This is very true. Very true. Well, no more true than what true is. 'Very' comes from French (via the Norman Conquest). It is cognate with French 'vrai' for 'true'. over the course of two hundred years after the conquest, there was an influx of huge number of Old French terms. 'Vrai' slipped over to mean true (foreign language usages are more likely to be 'repurposed' (i.e. misused). There is a bit of the etymological fallacy here, that the current meaning of a word should be what it used to be. There is no doubt that very means very nowadays. Maybe a little doubt in 1200AD in London.

But truly 'true' has the literal meaning of  'that which is the case'. Etymologically though it means all sort of things like 'faithful' or 'honest' and only became the opposite of false probably after 'vrai' became 'very'. So semantic drift happens, but that doesn't mean the drift was wrong or incorrect or bad or led to the downfall of civilization.

This is not to say that I like the non-literal use of literal. It hurts me (not literally) when I hear it used non-literally. It's so obviously intended to be meant literally and a non-literal usage just contradicts itself.

What is artificial intelligence?

Most people think of artificial intelligence as a walking talking robot that looks and acts almost human, and that this is the goal of AI, to make a simulacrum of all that a human does. But there are a lot of nuances to this, well, frankly a lot of major differences with what artificial intelligence really is.

I am writing this to clear up a lot of misconceptions that I read about AI, not to give _a_ definition, but rather to give _some_ definition to it.

So first, a first attempt. Artificial Intelligence is a computer method (rarely mechanical) that seems to do something only a human could do. A talking responding machine is AI, but a walking machine, with articulated legs, is not. Unless... well there it is, sometimes it is the application that is AI and sometimes it is the methods. And a chess playing machine....well is that AI or just really good engineering, exploring all possible game possibilities quickly?

We already have two nuances: how an AI is engineered, and what does the label apply to. The methods of AI engineering fall into two broad areas: human simulation, attempting to mimic the internal biological/psychological processes, and direct engineering, doing what it takes no matter what to make the engineered device act externally intelligent. A language parser uses scientific theories of linguistics to mimic how the brain is supposed to manipulate the text of a sentence to determine understanding. A chess playing machine uses alpha-beta pruning of deep game trees, contrary to the usual method people which is to look at only two or three moves deep and vaguely judge 'strength' of a position based on experience.

A lot of what was once considered AI is now considered plain old engineering. Once you see under the covers that the method used to look outwardly human is only a boring step by step recipe or lookup table, it loses it's 'wow' AI appeal and just seems like regular non-AI computing. in the 1600's, calculating machines were magical and might have been considered AI (if that were a thing then) because most people thought that people themselves doing it was magical much less a machine (OK this is a bit hyperbolic. what is 'most'? who are these people?). But you get the idea. A machine doing chess is magical (or has a small hidden turkish chess master manipulating the controls within). But Deep Blue in 1995, which beat Kasparov the chess grand-master, was pretty simple just using search trees, no ethereal silicon-embodied intuition ('just' = very rocket-sciency search trees and some hand-curated openings, gambits and end games).

An AI, with the indefinite article, is by popular usage, the thinking talking machine. Because of the potential for confusion the term for that nowadays is usual referred to as an AGI, Artificial General Intelligence.

But now to more substance. All along we were going along in vague style, not really knowing what AI really refers to concretely, relying on a presumed common idea of AI without really knowing. So here are some concrete examples of what AI is.

First, applications, which is what externally looks like everybody's general idea of AI

• vision - converting digital images into words 
• language - speech recognition, natural language processing and understanding, chatbots
• games - chess, go, crosswords
• problem solving/reasoning - word problems and puzzles, expert systems
• planning - taking a set of goals and initial conditions
• robotics - not the mechanical part of for example an articulated hand grasping an object but coordinating a path over a complicated landscape or separating parts on an assembly line
• learning and memory - most of computer science has been intent on solving the memory problem and has done it well. Financial statements, medical records, airline reservations, online libraries.

And then there's tools and methods, how it's actually done, the man behind the curtain.

• logic and other classical mathematical methods - for reasoning
• probability/statistics - most machine learning methods are within the realm of stats
• combinatorial algorithms - when a perfect algorithm for one domain works most of the time for another, then it's called a heuristic
• decision trees - chess? one big decision tree? Yes.
• optimization - specifically numerical algorithms for optimization like simplex or A*
• cognitive science - cognitive psychology, neurology (brain physiology, neurons), linguistics, so you know how it works in actual biology, so you can either be inspired by it (neural networks) or simulate it (parsing a natural language according to linguist's rules)

Apps and tools are orthogonal; an app can use different tools. So NLP is one application area of AI, and many tools are used by it: machine learning using large language corpora, and independently linguistic knowledge.

The man behind the curtain is an apt metaphor for AI because like in Oz, the apps look magical, but it turns out it is, well, not exactly charlatany; the methods are so...mechanical and spartan and inhuman and small tricks. For example, crosswords seems like you need to have not just broad and deep knowledge, but also clever ability with associative memory and puns to read behind the clues. The recent very successful AI methods for crosswords is just to use a dictionary search and almost ignore the clues themselves, by any means necessary no matter how cleverless.

With all that said, the motivation for this was what I find to be a lot of misuse of all these labels, one term being used for another. So here's a bullet point summary:

• AI is what you call the big show, all the different things together
• AGI (Artifical General Intelligence) is a very limited example of an AI app but the most salient, the 2001/HAL intelligent like spoken interface. This seems to be like the end goal of AI but currently unrealistic, or rather very realistic with very limited expectations.
• ML (Machine Learning), which is mostly just statistics, is a tool to quickly create AI apps from lots of data without having to rely on a domain expert
• ANN (Artificial Neural Nets) and DL (Deep Learning, which is just a big NN) is just (of many) ML techniques

A meta-note about NP-completeness and its current state of (lack of) knowledge of

Recently, Scott Aaronson wrote a short-book-length summary of the current state of knowledge about NP-completeness. This is an honorable tradition, starting with the classic and definitive book by Garey and Johnson, then Sipser, Cook, Wigderson, and Allender. Aaronson gives his spin on things (a little more on the quantum computing side) but hits well (like the others) all the great points: Turing Machines, reductions, completeness, the hierarchy of harder problems, what is conjectured to be the case (P != NP), etc.

What I want to do is point out the unsaid things, those things that are obvious to insiders and don't need to be said, but are inscrutable to outsiders. The outsider may realize it or be told it explicitly but not really get it understand it because, you know, math.

The most important thing is that, of all the things stated, all the theorems, all the papers and books, the great number of words and symbols written about the subject, all these things are written about what we don't know about NP-completeness. We don't know whether P = NP. We don't know whether NP = coNP. We don't even know whether Graph Isomorphism is in P (early 2017 Babai retracted 2015 quasi-polytime algorithm, just sub-exponential). We don't know. We don't know so much. Every day there are books and papers published that are essentially just outlining the boundaries of all the so many things we don't know about P vs NP.

Most of the theory surrounding P vs NP was developed in the 1970's. Lots of small results (with great effort!) have been established over the years. We do know now that NP = PCP(log n,1) (Arora et al. 1998), NEXP is not in ACC^0 (Williams 2011). No, not small results but proofs of things that we really expected to be easy. Some specific problems have been shown to be in a particular class unknown before: Primality is in P (AKS 2004),  LinProg is in P (Karmarkar 1986)  UGraph Reachability is in L (Reingold 2004), but one usually expects it (relatively) easy to find an optimal algorithm for a problem...it just  turns out that all these were very difficult to discover.

But all these are minor results. One-offs. They may spark a cottage industry in their subcommunity and are undeniably important. But they are... they are only making infinitesimal progress towards P vs NP. In fact, because of this meta-intractability (get it? the problem of intractability is itself intractable), no one actually is currently attacking the direct problem of P vs NP. Most of the (great amount of) work goes into ancillary situations, extending minor results, changing the problem, all in the hopes that there might uncover one little tiny bit that makes the difference. It feels like panning for gold in that you dig up tons and toms of material, wash it through multiple levels of gratings, wash it some more, refine the dirt, wash the dirt some more, and if you get a couple of tiny specks of non-dirt that just turns out to be copper, you call that a good day.

As far as history of science goes, this is a rare situation. Usually, a bunch of data is collected, some patterns are noticed, you collect a lot more data (and all this new data is useful and interesting), these patterns look like they repeat in different weird ways, then a consolidating theory (a scientific experiential theory that is) is formulated that generalizes and explains the prior data, and then a search is made for even more data which then either confirms the theory or encourages refinement or maybe even over-throw of the first theory. Similarly in math (which the P vs NP problem is a part of), a bunch of small theorems are proven, then some (usually a group but usually a single person gets good press) see a pattern, add a few definitions and prove a big theorem that combines them all, and then further consequences of that big theory are applied to more instances (note that yes, in math it is very rare (never?) for a big theory to be overthrown because a proof is a proof (right?) of course it is, but the refinement comes either in how many new more complicated problems to explore are created that go beyond the first theory.

Anyway, the rare situation with P vs NP is that we've essentially gone almost 40 years now with the same state of affairs of lack of knowledge. The progress that there is is not steady, and any gains seem underwhelming at least from the outside (the first outsider thought is "You're so smart, shouldn't you have figured that out a long time ago?"). Even the most minor seeming result is astounding (NEXP is a very intractable class and ACC^0 a very tractable one - how come it was so hard to show NEXP not in ACC^0?).

Also, back to more basic things. All this literature, all this theorem proving hasn't really established, hasn't really given a definitive proof of the pinnacle question P vs NP...but it's almost a scientific proof, a proof by having lots of examples in mostly one direction. All the reductions, all the intermediate problems, al the proofs of no proof by method X, are really supporting evidence (not mathematical definitude) just supporting the particular direction that P is a strict, proper, much smaller subset of NP, that NP problems are actually harder and we really can't find exact and fast algorithms for NP-complete problems. That is, even though we have 40 years of not proving one way or the other, it (almost) all goes in the direction that they're probably not the same.

Practicality vs Esthetics in DataViz

We want both. We want for something to be practical and nice looking as possible. We want the shiny doorknob to look elegant, but we also want it to work smoothly without having to jiggle it, and not to need cleaning all the time. Sometimes one has to be given up for the other, an industrial assembly line may be a little grimy all the time but it gets the job done; of course, cleanliness may be a desired property of the object and practicality and esthetics share a common cause.

Sometimes we want something that is esthetically pleasing and superficially practical but not necessarily perfectly practical, like a watch with no face numbers.  The esthetics is the desired intention.

That's all philosophy diatribe to justify something that bugs the crap out of me.

Basically wordles are the worst. And periodic tables (except for The Periodic Table which is the best). And usually subway diagrams (except for actual city subways). Here's one for semantic web technologies:

This! This is the worstest.

If you know what each of the entities are, and you make all sorts of qualifications, maybe this makes sense a little. It makes only the slightest bit more of sense if 'A above B' means A is 'built on B'. But then there are all sorts of 'If you did that, then why did you do that?' questions (why is encryption and signature off to the side only for some, why is logic and proof separate, is Unicode really such a huge important base technology, etc etc). Wait, isn't a namespace a particular kind of URI? There are many variations on the 'Semantic Web Stack', but each in its own way has all these "I don't get why they did that?" problems. This is all about esthetics (Nice color combo!) and little to do with imparting coherent information. No, you will not learn anything from this. Wait...what the hell is 'signature'?

Trust vs Depend

"I trust that guy as far as I can throw him"

"I can't trust him to complete the project on time"

'Trust' is used in two different ways. One is the usual opposite of falsehood. If you can't trust them, they are a liar. This presumes intent and is almost demonizing.

The other way is dependability. Trust of the outcome. If you can't trust someone this way, it's not a reflection of their evil intent but about ability to execute. This is very different from falsehood. You can actively do something about this.

The other way you can only seek other sources of information.

So instead of 'trust' 'use 'depend on'. 'Trust' makes it sound like you think they're lying. 'Depend' just means there is doubt without judging.

Testing Software and Machine Learning

Testing is a major part of any non-trivial software development project. All parts of a system require testing to verify that the engineered artifact does what it claims to do. A function has inputs and expected outputs. A user interface has expectations of both operation and ease of interaction. A large system has expectations of inter module operability.  A network has expectations of latency and availability.

Machine learning produces modules that are functional, but with the twist that the system is statistical. ML models are functional in that there are one-to-one correspondence between inputs and outputs, but with the expectation that not every such output is ... desired.

Let's start with though with a plain old functional model. To test that it is correct, you can test the output against all possible inputs to check if. That's usually not feasible, and anyway is in the more esoteric domain of proving program correctness. What is preferred is instance checking, checking that a particular finite set of inputs gives exactly the corresponding correct outputs. This is formally called 'unit testing'.  It usually involves simply a list of inputs and the corresponding expected outputs. The quality of such a set of unit tests relies on coverage of the 'space'. Edge cases (extreme values), corner cases (more than one variable in an extreme value), important instances, generic instances, random instances, etc. Also, any time a bug is found and corrected, the faulty instance can be added to the list to ensure it doesn't happen again.

An ML function is, well, but its name, also functional, but the expectations are slightly different. An ML model (the more technical term for an ML function) can be wrong sometimes. You create a model using functional directly corresponding inputs and outputs, but then when you test on some new items, most items will have correct outputs, but some may be incorrect. A good ML model will have very few incorrect, but there is no expectation that it will be perfect. So when testing, a model's quality isn't yes or no, that absolutely every unit test has passed, but rather that a number of unit tests beyond a threshold have passed. So in QA, if one instance doesn't pass, here it's OK. It's not great, but it is not a deal breaker. If all tests pass, that certainly is great (or it might be too good to be true!). But if most tests pass, for some of 'most', then the method is usable.

There are three major parts to creating a machine learning model that need to be tested, (where something can go wrong and where things can be changed): the method or model itself, the individual features supplied to the model, and the selection of data. The method or model itself is the domain of the ML engineer, analgous to regular coding.

I can almost go so far as to say that testing is almost integrated to a large part already within ML methods. Testing use systematic data to test the accuracy of code; ML methods use systematic data in the creation of a model (which is executable code). And so if an independent team, QA or testing, is to be involved, they need to be aware of the statistical methods used, how they work, and all the test-like parts to the model.

Let's taking logistic regression as an example. The method itself fits a threshold function (the logistic function) to a set of points (many binary features, one continuous output feature between 0 and 1). Immediately from the regression fitting procedure you get correlation coefficients, closeness of fit, AUC and other measures of goodness. There are some ways to improve the regression results without changing the input data, namely regularization (constraints on the model), and cross validation. For the features, mostly independent of the method), first there are the number of features, how correlated they are, how individually each feature is predictive; each feature could be analyzed for quality itself. And last for the selection of data, also independent of the method, there's selection bias, there's separation into training, validation, and test sets.


Where QA can be involved directly or indirectly:

- ensuring metric thresholds are met - analogous to overseeing unit-test coverage of code)
- questioning stats methods (like being involved in architecture design
- cross-validation - both makes the model better (less overfitting) and returns quality metric
- calibration of stats methods - quality of prediction
- test and training data selection - to help mitigate selection bias
- testing instances - for ensuring output for specific instances (must-have unit tests)
- feedback of error instances - helps improve model
- quality of test/training data - ensuring few missing values/typos/inappropriate outliers
- UX interaction of humans with inexact system - final say - does the model work in the real world via the application, has interaction with people shown any hidden variables, any unmeasured items, any immeasurables, any gaming of the system by users.

The latter seems to be the most attackable by a dedicated QA group that is functionally separate from an ML group, and all the previous ones seem to be quite on the other side, only the domain of ML to the exclusion of a QA group. But hopefully the discussion above shows that they're all the domain of both. There should be a lot of overlap in what the ML implementers are expected to do and what QA is expected to do. Sure, you don't want to relieve engineers of their moral duty to uphold quality. The fact that QA may be looking over quality issues doesn't mean the data scientist shouldn't care. Just as software engineers doing regular code should be including unit tests as part of the compilation step, the data scientist should be checking metrics as a matter of course.

Notes from China-Japan trip

The following are my unedited notes taken on my iPhone on a two week trip through China and Japan: Beijing, Chengdu, Yangshuo, Honk Kong, and Tokyo. Reading through them I notice a few autosuggest errors, but also some that look like autosuggest errors but are really intended, just me being weird. I also notice that there is quite a bit that is not here, either because we were too busy doing things for me to type up, or because I didn't think of it as a thing to write, despite my memory now realizing it was a thing.

The actual itinerary was:
Day 0,1 Fly from Boston to Narita, Narita to Beijing, taxi to hotel
Day 2 Beijing morning - Tiananmen square, early aft - lunch, late afternoon Acrobats/magic show
Day 3 Beijing free day, walking around SanLiTun, local hu tong
Day 4 Beijing morning Mu Tian Great Wall aft/eve flight to Chengdu
Day 5 Chengdu morning Jin Li St, lunch Szechuan cooking, aft People's Park, evening Chinese opera
Day 6 Chengdu morning Panda Zoo, afternoon Chengdu to Guilin flight
Day 7 Guilin to Yangshuo, morning drive from Guilin hostel to Yangshuo, aft Li river cruise
Day 8 Yangshuo morning - bike ride along rivers, climb Moon Hill arch
Day 9 Yangshuo to Hong Kong travel
Day 10 Hong Kong morning - Victoria Peak, aft - Peninsula Hotel high tea
Day 11 Hong Kong, Lantau Island morning Buddha, aft - Tai O fishing village.
Day 12 Hong Kong to Tokyo flight
Day 13 Tokyo - WTC and bus/boat tour
Day 14 Tokyo - Joypolis amusement center
Day 15 Tokya to Boston flight

 ---

China Japan

Day 0 (Jun 25-26)
Flight
A little more room than domestic
Ian port asks to trust. Cables everywhere.
Two people in back cabin w face masks. 3, no 4
1/2 of area of plane is 1st and business class
Already annoyed that I can't look up what actor that is in imdb

Day 1 crosses the date line in flight. Lots more flying.
Narita airport
By the time at Beijing airport I think I've seen every possible JAL entertainment option. Every movie even Japanese and Korean ones.
Fried rice at place downstairs from hostel.

Day 2 Beijing
Tian an men square huge but still people everywhere.
Women wear more dresses. Men spit. Some People sit on their ankles. Iranian and Nigerian tour group.
Guys rolling up their shirts to cool off. Ugly
Among the throngs of tourists. After TAM lots of hawkers but also lots of people asking for money. Guy with one arm
Restaurant not organized looking.
Little girls all dressed up
Big areas, but then smaller garden areas very nice.
Young Woman w cat ear head band
So many Chinese tourists. I thought I was special.
Breakfast bao zi, jiao zi, banana
Lunch kung pao chicken, chao Cai, vegetables, frog legs.

Bathroom at hotel has an toilet w water saving two flushes. Shower to open floor.

Dinner Peking duck, cabbage. Cauliflower, salt duck bones

Day 3
Breakfast at hotel. Meh.
Complicated subway trip to sanlitun mall area which ended up everything we wanted to see being closed. Complicated trip back. Raining hard by the time we got out.
Lunch at place across st duck and fried noodles. Really good.
Ping pong in public
Lao Beijing, I heart BJ, lots of public spitting by old men, clear g of throats
Small groups wearing monks or karate clothes
Electric mopeds that are entirely soundless

Day 4
Breakfast at place next door.
Road out of Beijing lots of nice new and old skyscrapers buildings. Plants decorating highway.
Cars look identical
English/pinyin everywhere almost always.
Everybody has an iPhone (everybody, or Samsung)
Lexus dealer
Road out of city is all tree and shrubs growing for sale.
Closer to mountains, very touristy but not many tourists.
Lots of isolated shrines (serious well decorated) just sitting in a field or on top of a small ridge

Great Wall - amazing. Someone forgot a level when building

Lunch - subway. Tastes the same everywhere.
I saw grape leaves growing. I smell... Cows? Pigs?
Lots of brightly colored exercise equipment next to playgrounds
Two camels sitting in bright green river basin. No water.
Our bus driver really likes passing everybody no matter what. He might slow down or speed up if a car comes the other way.
Roof top water heaters.
Everybody growing a very small crop of corn outside their walls
Gas stations seem empty even though highways full of cars
More people smoke but not everywhere. Female truck drivers

Dinner on airplane - measly Lang pao chicken. Seaweed salad muffin yogurt-thing sweet rolls (called bao zi)

Day 5 Chengdu
Breakfast baozi rice porridge bean sprouts savory flan
Jin li st cicadas yak meat rabbit heads, crossbow

People's Park-100 things

Cars go one way on one way streets but everybody else does what they want. Nobody is driving crazy but lanes seem optional. Safe but worrisome. Not speeding at all just questionable choices and behavior by cars, bikes, peds all.

Stores - 100 things to say

Chinese opera -
Taxi cheap
Cheng du fancy, so many new buildings, luxury stores
Post-modern story within a story
Mask changing
Grating female opera voice,
Very traditional music and dress mixed with very modern music and effects.
Chinese english Korean Japanese subtitles
San guo, 108 stories

TV - stupid, just like US
Motorcycle repair on the sidewalk.
People tend to try to speak minimal English if they can.
Skyscrapers go on and on.

Day 6 Thu - cheng du Pandas
Tibetan restaurants/stores
Lots of subway stops
Business skyscrapers give way to apartment skyscrapers. I see a spot where they missed.lota of construction. Lots of mopeds
Cicadas at panda zoo
You want to hug a panda

Lots of kids with shaved heads but a heart or circle.

Huge outdoor mall area as first couple floors of skyscrapers

Taxi driver left us off a block away and then continued on past front of hotel ahead.

You can see the sky in Chengdu (not in Beijing)
People walk across the st without a care with cars coming on either side.
Roads have lanes for bikes/scooters.
Free condoms in lobby of hotel

Day 7 Guilin to Yangshuo
Raining
Potholes
Beautiful streets in Guilin (riverside hostel really nice along river)
Guilin dirt is red
Everybody has a smart phone
Banks everywhere
Motorcycles and scooters in rain doesn't stop them
Rice
Constant change in crops. Beautiful mountains
Patchwork crops

B landscape totally flat but scattered with limestone hills

Nice cars from everywhere. Volvo Skoda Mitsubishi BMW



Etc toll ezpass

Yang shuo new built buildings with no siding look abandoned in the middle

Li river cruise amazing. Pictures are a weak copy of reality.
Trash in water at harbor
Selling dried fish onboard
Taking pictures of the 20 yuan with li river landscape on bill and in background.
tourists are almost entirely chinese (again I thought I was special)
Talked with older couple about kids. A younger man tried to talk (in sorry but I'm shy. The language thing or the social situation)
Water buffalo, bamboo, edging on river, cormorants, houses along river,

Day 8 Yangshuo
Breakfast - noodles

Raining off and on humid

Bike ride/moon hill - too much to see
Rice paddies, continuous cultivation of everything, small villages w new concrete roads, poster with alternative medicine flames, dams, pet dogs not saying hi, fruit trees, melons, grapes, lotus, wildflowers, Palm trees bamboo exercise parks other tourists, mao shrine in house. Talking with tang up mountain steps


Moon hill natural arch - easy short climb through bamboo etc forest so muggy can't tell if rain or sweat.

Lunch - Ian said best yet, taro chips, peppers not as spicy as claimed, egg and tomatoes, potatoes, beef w onions and veggies

Driving - people drive very slowly. But don't seem to follow right of way, or rather they do but not lines on street. Horn honking is loud but meaning is polite, to tell you that I'm about to pass

Got a lift with a motorcycle 3 on. Totally safe except very questionable
Truck filed with dogs of same type
Shopping in Yangshuo - 'local market' of touristy things snack aisle had all sorts of vacuum packed fish or meat things, chicken feet.

Dinner - Nolan fried rice, Shohreh fast food buffet. Ian spicy noodles. Faux tiramisu, Black Forest cake, bowls in plastic vs buffet confusing.


Day 9 travel Yangshuo to hk
Yu long river
Building everywhere
Potholes!!!
Cell usage everywhere
Shohreh says How dare they have tolls on roads in China
G65 off 543
Less pinyin in guilin, lots in Yangshuo
How did hills in guangxi develop?
Xanadu Putao 535
Gas station teardrops on highway Sinopec
Hills go on forever in every direction
Little terracing of rice paddies I guess because everything is so flat between the hills
Scooters and motorcycles with umbrellas either attached or holding
Highway 2 yrs old
Bamboo scaffolding looks flimsy
Highway sparse
Red dirt
Er 508 exit near G72
Tree planting in grids like France
Rumble strips 3 3 2 2 1 1
Now lots of hills far off in the distance
Cloudy consistently so no sun, have to use compass
Hard to balance typing and looking
Hills shorter and less steep, connect together
Individuals working in fields not so quaint. Hard work
Cars mostly new. Mini work trucks look much older
Gravestones at base of limestone hill
Exit s22 01
Store fronts entirely open everywhere we've been, even large hardware groceries toys motorcycle repair
Shrubs along highways
Wide bike scooter paths along main roads

On train to Shenzhen
Little villages nestled within tall hills. Lots of new bldgs they don't want to put siding on many, some dirt roads
I see a non-chinese person and want to stare.
Big empty train platforms (Guilin was full)
Guangdong flat broader bigger fields more taller bldgs
Large ponds with pumps in the middle
We had the kindergarten car. So many excited singing kids

Shenzhen metro
Little kid outed me as meiguoren
Littler kid has uber alles t-shirt

HK
2 customs
Ian says why do they need the same info that's in the passport copied onto a form?
Great air conditioning
So many mountains on the mainland side of HK

Jordan - like NYC smashed into 10squarr blocks. A seller in the night market swore at a customer for walking away. Great Japanese candy store.
Pakistan halal, Thai Vietnamese, doner kebab.
Multicultural: Malaysian, African Indian, southeast Asian.

Day 10 HK
Breakfast dim sum fulun restaurant
Cruise ship, modern mall
People are mostly very curt ad humorless
Skyline of HK island
Lunch at peak restaurant (noodles, turkey club, fried rice)
Peninsula hotel tea
No dinner (Ian had fried chicken, cucumbers, noodle soup) I had milk tea.

Day 11 HK
Bfast - McD's
Metro real nice try
Lantau islandvoiin
Cable car to Buddha - enlightening
Bus to Tai O fishing village - bust, false advertising. So many squid related food products.
Ian wonders that people still love like that.
Drivers are 'normal' here, drive faster, follow rules
Scenery great
Infrastructure for rural env really modern. Feels European.
Squat toilets ok w shorts.

Lunch
Subway (I had mango lassi and samosas from Ebeneezer's)

Dinner
Fulum fisher mans wharf restaurant
Yao had us wash our glasses and bowls and cs
Waitstaff surly and miffed by other waitstaff, walk away in the middle of the order
All mostly retired peopl

Misheard: ff7 sadder than titanic

Had laundry done at a chinese laundry

Day 12 HK to Tokyo
Bf cheap buns
Lunch on Cathay pacific al, sns pork
JR Narita line
Japanese rice fields much neater larger and fuller than Chinese
People dressed less casually (not formally at all) wait... It's rush hour, everyone is coming back from work.
A handful of masks


YMCA br room very 69's looking
Guy with masked sneezed terribly in elevator

Searching for sushi conveyor place we ended up at a ?hibachi? place

Dinner Sukiyaki - the travel shows never showed that at some places, pay at machine

7-11 had mostly people looking for convenience dinner

Day 13 Tokyo
Breakfast YMCA turnip/fish broth soup rice kimchi mini mushrooms
Always pay somewhere else first

Lunch at Hamamatsucho bus station cabbage salad

Everybody dresses the same. Everybody. Black pants, white collared shirt black or blue business suit. All work people.

2 Indian restaurants

Bus tour - 12m Tokyo, 30 area, 127m country
WTC
7-11 covers Tokyo too
Convenience stores, small restaurants everywhere
Clean clean clean
ATMs have english, but lots of places want cash or don't take
cc
Few bikes
Very modern concrete and steel bldgs (not much mirror siding)
Tour fact: most people follow both Shinto and Buddhist practices/beliefs

It raining constantly but people (and tourists) are still out.

Trains on time to the minute, and scheduled often

Chinese restaurants

Dinner at sushi manemura conveyor belt super excellent and cheap kind of loud 'not crappy'
People talk in restaurants, not on pub trans
Mistook tea for wasabi powder. Mixed with soy, really not the same

Review of Hanemaru ( in the Kitte building across from Tokyo station):
Conveyor belt sushi is a crass gimmick, like sprinkles on donuts or strobe lights at fireworks show.
But there's also the accompanying atmosphere and quality of the sushi. We were foreigners here but the conveyor belt was very practical for us, just take what looks good. Everything looked good.

And the war zone that was our table at the end with random stacks of miniplates and dribbles of soy sauce, all of us stuffed beyond imagination, only $15 a person. Luxury eating at regular prices


Day 14 Tokyo
Breakfast seaweed soup at y
Joypolis
Huge landfill developed area full of buildings k.

Urinals have video games, measures volume

Every so often a woman in a kimono or a young couple (just married)

Kabuki theater is such kabuki theater

Every joke has been made before

Dept store basements have the food shops

English (not just roman) on all transit.
Joypolis had written english instructions. Taxi drivers and info people and waiters very polite about no japanese speaking

Dinner selection of foods from food shops.

Day 15 Tokyo to Boston
TV shows have a small box in lower left or right of someone watching the same show and giving their reactions, just facial expressions

Cars just have a slightly different body shape.

Commercial Parking lot with space for two cars. That's it.

nobody has driers, everyone hangs laundry to dry

'Blonded' hair

House rooves some regular some 'chinese'. All small and shallow.

Taking pictures on a train is hard. By the time you realize there is something interesting to take, it's already gone.

Street and rail crosswalks everywhere and all used

I am what I eat or don't eat on purpose

I grew up in Richmond, VA, a southern town but not deep south. As I grew up (1970's suburban white), I never felt southern (or northern, or anything), and people outside of Virginia asked my why I didn't have an accent. I don't (sez I!) but every so often I'll let through a "y'all" or "hunnert" (for 'hundred') and think it is the height of articulate speech to say 'Et-lanna' for Atlanta (I just say 'lanna'). And as to food, I grew up on McD's hamburgers and Taco Bell and HoHo's and DingDong's and bagel and baloney sandwiches and frozen mini pizzas like everybody else in every suburb in the US. Sure, there were a noticeably greater amount of pine trees, below the top two inches of topsoil the dirt was orange, and I got chiggers (once, and only when I came back to visit as an adult).

But I now realize there were a number of things I ate (or decidedly didn't) that say 'Southern':

• barbeque - I never liked this as a kid but love it as an adult. The only experience of this I had as a kid was a drive up that my mom liked. I got the burger. But now I really appreciate it.
• cornbread - The kind I had as a kid was dry and dense, almost peppery, coming in small fingers made from a metal indented pan to make them look like corn cobs. What I consider edible cornbread now is almost cake, sweet and thick.
• greens - collard greens, mustard greens, turnip greens, creamed kale, I loved them all, cooked with with salt pork.
• fried chicken - my mom was into health and stuff, so she didn't make it with batter or skin. Popeye's is the best ever, but you can't have it too often without having a piece of your heart die. Literally.
• okra - never had it as a kid and rarely as an adult. Some people hate it and some love it. I don't care.
• brunswick stew - the best way to burn your tongue. Great from a can or made from scratch.
• mac and cheese - never had it as a kid (did I mention my mom was into health and stuff). I still don't understand what's so great about it. But this is southern?
• watermelon - it was around -all- the time as a kid. It was OK, but made a sticky mess, sort of like cotton candy but wet. also seeds are a pain. I'm still ambivalent about it
• Krispy Kreme donuts - like Kleenex, these are the canonical donuts. But I didn't care for them. Again a little messy. DD donuts are real donuts. Sorry, South.
• grits - hell yeah. As a kid I had the instant kind where you add boiled water, which if you add the a little grated cheese and hot sauce are great (ooh and a boiled egg).  Waffle House and diner grits have so much extra ... something... that they feel like something else. Not bad but just ... too much.
• roasted tomato halves - best thing ever at fancy buffet events (after all the others). Fried green tomatoes is just weird like some foreigner thought it would be a good idea for the title of a quirky movie, and then people thought it was an actual thing. It's not a thing. Roasted (red) tomato halves are a thing, a real thing that is actually good.
• sweet tea - ugh...who would put that much sugar in their tea? I mean, unsugared is a little too ascetic but sweet tea? ugh.
• crab cakes ... ugh.
• pecan pie - that's southern? I thought it was just pie. Anyway, banana pudding kicks its ass. Not that pecan pie is bad. Just if you had to choose.
• fried pickles - ? hunh? That's a thing? And a Southern thing? I've never heard of that. Ever. Why try to gild the lily?
• hush puppies - mmm. Only ever had them on vacation, Virginia Beach or Nagshead at a seafood restaurant
• sweet potatoes and yams  (there's a difference?) - ugh. These things look funny! And taste funny! No way! Potato or sweet potato? Take the potato.
• tofu - a versatile basis for any... haha, just kidding. That stuff is so bland, you add it to dishes to take away taste. Also, not Southern.
• pig maw - eww. pass.
• black-eyed peas, lima beans, just beans - I avoided these every New Year's. Ugh. It's not that they taste bad, it's just that they don't taste good.
• biscuits - these are southern? They're not universal? I have nothing against them, they're OK (when not greasy)
• chitlins - that's just a funny word that people use on old west shows. Nobody actually eats that stuff. 
• smithfield ham - as a kid, tasted funny, why would you ruin a biscuit sandwich with weird meat. Now I appreciate it a lot.
• apple butter - if this is at all a southern thing, it's great. It's great anyway.
• deviled eggs - also, this is southern? They're great, but don't have too many, especially if they've been lying in the sun

Some of these items were off the top of my head. Some I was reminded of by the wiki list. Those on the list that are not here I just don't recognize, didn't think of as southern, or just don't care about. Also, I find it funny to go to a 'soul food' restaurant and wonder what's going on because it's just food.

Effect size versus statistical significance

One of the major tropes in the p-value wars is the difference between statistical significance and effect size. The usual (important) observation is that you can have a calculation on data that results in very small p-value, meaning very high statistical significance, but very small effect. And often this can be effected by increasing the number of instances: the more instances the smaller the p-value can be guaranteed, that the phenomenon is really not due to chance, no matter how small the scale phenomenon actually is. This is not to say that the phenomenon is not real, just that the phenomenon doesn't change that much in one direction.

This difference is presented often laconically ("Using Effect Size—or Why the P Value Is Not Enough") as:

Statistical significance is the least interesting thing about the results. You should describe the results in terms of measures of magnitude –not just, does a treatment affect people, but how much does it affect them.(Kline RB)

This makes it sound like you have two things that can be presented, and one is much more important than the other. But it's a false dichotomy. You want both. The magnitude is descriptive stats - how big it is. In an experiment on n individuals, fish oil tablets increased memory performance by 10%. If you don't know the effect size, what exactly beyond 'better' do you know about the phenomenon? Statistical significance is trust - how (mathematically) representative the sample is of the population. You can claim something is better but can you really trust the claim?

It's very easy to see how to manufacture a high statistical significance but low effect size - increase the number of instances. In fact, as you increase n, almost all statistical tests asymptotically approach statistical significance (for real world phenomena). Chi-squared is the worst!

A consistent high effect size (over samples) leads obviously to high statistical significance.

But it is possible to have high effect size and low significance.

So in the end, it is not one or the other. Both should be presented. The effect size tells you how different the sample shows phenomenon is, and the p-value tells you how much you can trust the sample that showed the phenomenon.

Annoying Sciency Tropes: Big effing number

Here's a really annoying pattern that comes up in science journalism (OK really any news story that involves a number) and it involves so many fallacies and misdirections and insults to intelligence that I can't over underestimate it. It is the presence of a Big Number.

The yearly output of carbon dioxide gas into the atmosphere is 50 bajillion tons. Wow, that must be bad because a bajillion is a lot. (Also 'tons'. You can have a ton of air? (of course you can that's physics, but it is counterintuitive enough to simply leave the reader with the simple incoherent feeling of 'wow').

The number of deaths due to the Iraq War of 2003 was approximated at 600,000. Of course that is terrible (any such death is terrible). But is it reliable? Is the scale right? How was the number arrived at? What groups are in that number? Is it overcounted? Undercounted? Adding a zero hardly changes the impact of the story but is still wildly inaccurate.

Million, billion, trillion are hard to distinguish. They're mostly 'really a lot', 'really really a lot', 'that sounds like a lot'.

I realize I'm giving these without context, but the point is that often news stories lack all context too.

There's a little bit of technical obscurantism going on (is a nanometer bigger or smaller than a picometer?) which expects education; that is, it is questionable whose fault this is, the one using the technical term or the one reading it. If the reader were educated, this is the best most accurate communication, what technical language nuances are created for. If the reader is not educated in these nuances (which are not nuances to the initiated), then what?

Part of the annoyance is that this is usually combined with a Base Rate Fallacy; usually no comparison data is given - no comparison with the total or comparable items, no context. For example, the debt of the US government is given (latest number) in news stories as $14 trillion. Obviously this is a big unfathomable number, but also there is nothing to compare it with, either the historical debt (what the trend has been over the past few years, what the debt in other countries is like).

What's the solution? For the reader, look outside the article for the base rate or trend. For the writer, supply that! Give something to compare with.

Bullshit

Bullshit. There's no better word for things that sound true or plausible but have no connection to or no support in reality or any attempt by the speaker to make that connection. The metaphor is weak but it captures the feeling of the realization about what someone else has said.

Usually a statement is called bullshit if it stretches the bounds of plausibility. If it turns out to be a falsehood, it is considered a lie. Harry Frankfurt wrote an entire book on the subject, trying to solidify it (ugh, metaphors) as a statement by a person who is not intentionally trying to lie but rather has no concern at all for its truth value. That is, a bullshitter doesn't care whether a statement is true or false. Frankfurt is a philosopher and is trying to shoehorn a word into his own internal concept or into one that is more logically amenable, which is to say I think that usage is charitable. A bullshitter is trying to lie; if the statement turns out true, they would be surprised.

But the label is problematic. Bullshit is a little taboo (or a lot given the context). What are the alternatives? There are many but they have their problems, too.

All the following words fit the syntactic pattern, in response to something said, "That's X".

The first category is the anachronistic nonsense words:

• balderdash
• bosh
• bunk
• bunkum
• claptrap
• codswallop
• fiddlesticks
• flapdoodle
• folderol
• guff
• hokum
• hooey
• malarkey
• piffle
• poppycock
• tosh
• tommyrot

What do any of these words mean? They're entirely opaque nonsense words, made up by someone long ago out of random sounds to sound like what they're describing. Another thing they all have in common is that currently (and for the past fifty years at least) no one in their right mind would utter these in sincerity, given that they sound like an old man smoking a cigar mincing an oath. All of these words are idiomatic, inexplicable. This description is almost the meaning, which may be the originally intended psychological effect.

Moving closer to reality, the next is the most populated category (and likely most thought of if not used), the actual shit category. Shit itself really isn't (metaphorical) bullshit - metaphorical shit is just worthless, but metaphorical bullshit is a damned lie. Many are minced (crap for shit), but at least they are analyzable.

• bullshit
• crap
• crock (of shit)
• horseshit
• horsetwaddle
• horsefeathers - minced horseshit, so to speak
• twaddle

Now there are real, relatable but still not literal, for mixed up messes, trash or sausage metaphors:

• garbage
• hogwash - sounds like nonsense to me but actually is pig slop.
• rubbish
• baloney

Actual literal non-onomatopoeic words that are close in meaning but just not quite bullshit and still somewhat old fashioned:

• blather
• bombast
• drivel
• trumpery

And finally the list of actual, literal words:

• nonsense

Of all the lists of related words I've seen for bullshit, 'nonsense' is the only direct literal word for it.

And that is the real problem here. Bullshit is not nonsense. Nonsense is words that make no sense. Bullshit makes sense, may be true or not, but may be misleading. Word salad (a salad made of words), gibbering (of an idiot), unconnected train of thought (of someone distracted) are all nonsense. Bullshit makes sense entirely. Just the intention (or reception) is different from usual truth valued statements.

Therefore there is no good alternative to bullshit.