Meetings are the worst

If I had to identify, in one word, the reason why the human race has not achieved, and never will achieve, its full potential, that word would be 'meetings'.- Dave Barry

There are different kinds of meetings. Some are to make other people aware of what has happened, some are for making a decision together, some are for getting things done, some are for brainstorming, some are highly structured, some are for meeting people you haven't met before.

Sometimes you can just show up at a meeting and wing it if called on. Sometimes you have to be prepared for a meeting. Either way, for some time before the meeting, whatever you're doing, you're thinking in the back of your head 'argh, a meeting, what will I say about progress on widgets?'.

Who likes meetings? The ones who call them. Organizing and leading a meeting makes you feel important. It's like a club, an exclusive club that other people aren't in. "Why wasn't I invited to that meeting? That affects me! I would have liked to say something!"

I find that the proliferation of meetings themselves comes out of a more modern desire to be considerate, transparent, and to be inclusive. You don't want to annoy non-key players. 

But it seems like there are always too many meetings. "My schedule is full, never time to actually do anything."

Some people are primarily managers and some primarily doers. A manager's job _is_ communication so their job _should_ be meeting with people. A doer is doing something (engineering the deployment, tweaking the spreadsheet, reminding clients about payments, making the training video). See this blog post:http://www.paulgraham.com/makersschedule.html. And they should be spending most of their time doing. Communication is necessary, but it cuts into time doing things.

Also, we're all a mix of these two. You may be a CEO and need to talk to the other execs, and clients, and investors, and give presentations. But you also need to prepare that ppt and practice your speech. And even if you're the assembly-line widget-53-orientation-setter-class-3, you still have to spend time discussing issues with your neighbors off-line.

Consider these tactics to make meetings shorter/unnecessary:

- have a clearly stated agenda (even as a document) with clear goals or if it is intended to be a discussion have boundaries and stick to them. An amorphous or non-existent agenda just means people will talk for an hour with little outcome.

- don't be shackled by hours and weeks. modify the schedule to once every two weeks, or just 1/2 hour (or both, or less).

- always try to end a meeting early (don't feel like the time has to be filled out). No one ever leaves a meeting early thinking they wished it could have gone on longer. If the end of the agenda has been finished, don't stretch it out, it's done. If there are any further questions. 

Some people spend all their energy trying to figure out when is the last moment to jump in with an idea or question and it may come at the end.

- assign a non-key participant to be a facilitator (this is difficult to get, to perform, and loses that person's time)

- stick mostly to guidelines but be lenient. If a talk veers off the agenda (a problem is being solved but the meeting is about discussion), then schedule a meeting for the problem solving -or- be open about modifying the agenda (just stating an agenda item is often enough and details haven't changed).

- someone, not a key-player, should take notes (could be facilitator)

- record action items discovered.

These last few tend to increase the energy spent on a meeting. They're intended to increase the worth of the meeting. If a meeting's importance doesn't call for this prep, then maybe it doesn't have to be that long or have that many people.

"Probabilistic programming languages" aren't

I'm looking for truth in advertising: "Probabilistic Programming Languages" or PPL is a recent term to describe a very useful new paradigm in statistical modeling computing... but it's not a set of new languages. It is much more restricted to very particular kind of statistical modeling as opposed to the broad sense of probability.

As someone who likes a little consistency in language use, for words to have meanings you can mostly rely on, I am bothered by this usage (just as I'm bothered by the similarly mystically enticing marketing term Deep Learning). Here is a  very representative description of PPLs:

Probabilistic Programming (PP)
There’s a revolution in Computer Science called Probabilistic programming (PP) where programming languages are now built to compute with uncertainity in addition to computing with logic. This means that existing programming languages can now support random variables, constraints on variables and inference packages. Using a PP language, you can now describe a model of your problem in a compact form with a few lines of code. Then an inference engine is called to automatically generate inference routines (and even source code) to solve that problem. Some notable examples of PP languages include Infer.Net, Stan, BUGS, church, Figarro and PyMC. In this blog post, we will access Stan algorithms through the R interface.

from a blog article on PPL (which also tries to introduce new but uninformative terminology, MPML).

I expect words to mean things, and despite liking metaphorical usage in literature and expository writing, not calling a technical thing what it is sounds too much like slimy obscurantist marketing practice. If it is misleading in any way, it is suspect. Suspect maybe not in venal terms, but more likely suspect in intellectual depth.

For the record, the difficulties in the passage above are:

• There's no revolution, not in computer science, not in programming languages, not in AI. Maybe there's some recognition that there is some progress in usage, but it is incremental.
• No new programming languages are being built. No programming languages are being modified to accommodate new probabilistic data types. This is the biggest clunker. There's no new programming language thing at all. What is new is packages or libraries or functions, in the existing programming languages. PyMC is a library written in Python, and used in Python as native Python. Stan is written in C++ but it is not a syntax/semantics, just a library that is accessible from existing languages (R, Python, Matlab, Julia, etc).
• The idea of operating on distributions as a type is not actually new. Mathematica and Maple have had object oriented implementations of distributions, allowing operating on those distributions functionally. What these PPL packages add is approximation algorithms to compute values for Bayesian inference using Markov-Chain Monte-Carlo (MCMC) , which is fancy talk for calculating a number approximately. Pretty much very analogous to computing a p-value.
• All these PPLs are just library add-ons to existing languages. So in that sense don't worry that you have to learn a new syntax. You surely will have to learn how to use the library.
• It's not about probability in the large. Most all languages have probabilities already (restrict floats to the range 0:1). Some people are creating packages that make it more easy to use probability distributions (which some languages already had libraries for), and to manipulate those distributions (and make statistical inferences from them. But, no, it's not a revolutionary new alternative to languages with logic using probability. It might be a revolutionary library of functions that will make manipulating and computing with distributions and models easier, nut it's not a new language.

A programming language is a syntax and semantics and a compiler. A library is a set of functions written in a programming language usable by programs written in that programming language. One language can use functions written in another programming language as long as there is an interface.

To call a programming language probabilistic (or extension to an existing language that would render in my eyes the extension to be probabilistic), there would have to be a basic data type, like integer or boolean, that corresponds to a probability distribution (for the purposes of efficiency in compilation).

This is certainly a diatribe about naming. I have no qualms about the use of these PPL libraries. It's just the name. Distributional Modeling Libraries may be more accurate and doesn't have the same punch but is not actually incorrect like PPL. But if you're advertising faster-than-light speed, those words come with a lot of meaning, and should actually provide that without a lot of qualifications.

Words that are almost identical in spelling and meaning but are not

Some words have a huge semantic overlap that there's hardly any distinguishing them. Bucket or pail? Is there any instance where one of these is right and the other obviously wrong? (probably, but rare).

Some some pairs of words are really hard to differentiate but they have distinct meanings, if only you study the dictionary. They have considerable vague overlap, but a verifiable situations where one or the other just don't apply technically.

First in a series:

tragedy and travesty:

That poor dog dying. Was it a tragedy or a travesty? It could easily be both. A tragedy is a particularly bad outcome. Most everybody dead at the end of Hamlet (sorry, spoiler!); that's a tragedy. A travesty is a horribly distorted copy of something. The elementary school play of the Cherry Orchard had no sense of anomie; that's a travesty but not really tragic. A travesty is often tragic; a travesty of justice ended up sending the falsely accused to jail resulting in the tragic demise of his entire family. There are tragedies that aren't a travesty. Most people dying are tragic, but really that's sort of the order of things eventually. There are travesties that aren't tragic; that elementary school play. Tragedy: Hamlet. Travesty: School play

Irish Words in English

Irish is obviously overshadowed by English culturally. So there are lots of vocabulary items in Irish that came from English. But it goes the other way too. There are a handful of terms that were borrowed into English from Irish: the Saxon overlords ('Sasanna' = England in Irish) picked up a few things from the Celtic underlings (pronounced 'keltic' with the Irish 'C').

Yes, in the following even though there is a link, some liberties are taken. So maybe it doesn't come directly from Irish but via another language that eventually borrowed from Irish. Or maybe. Or maybe it was just sort of maybe made up to sound Irishish. But most of this holds up.

banshee  From Irish for 'screaming spirit'. Haha, no that's what it is in English. In Irish, it is 'bean sidhe', meaning fairy woman.

britches  From Irish for pants. Well, not exactly, but it is the Irish word for pants so that's good enough for me for it to be Irish origin. 

brogue  An Irish accent, usually in the phrase 'an Irish brogue'. 'brog' means shoe in Irish, so they must be especially noticeable kinds of shoes to have turned into a symbol of Irishness.

clan  From Irish 'chlann' children, clan

clock  From Irish 'clog' clock. A tortuus path through other languages with Irish involved.

galore  From Irish 'go leor' enough, a lot. Often paired with whiskey. But works for anything in English.

hooligan  yob, rowdy young man, From ... well it's probably made up from Englisher's mispronouncing the surname 'Houlihan'. Like all Irish have the last name Houlihan. 

phony From Irish 'fainne' for 'ring (for your finger)'. Presumably costume jewelery.

pet  From Scots Gaelic 'peata' for tamed animal.

shanty Official sources say it's from French 'chantier' for cabin, but there are claims for Irish 'sean tig' for old house (which is not good Irish grammar). Fine, but it has that Irish sounding '-y'.

shenanigans  From ... probably also made up to sound Irish. But I count it!

slew From Irish 'sluagh' a big number of people

slogan From Irish 'sluagh-ghairm' for war cry (by a big number of people) 

smithereens  From Irish 'smidirin' small fragments    

spree - From Ir 'spraoi' for spree.

trousers From Scots Gaelic treu for pants. Scots counts.

whiskey  From Ir 'uisce' water. In Irish, whiskey is 'uisce beann' or water of life.

You'd think there'd be more!

What's most important for success in X?

On a very particular Q&A site, I saw the question "To what do you attribute your success?". A very broad question, intended for a very narrow audience, but any answer should apply generally. Many answers say something like 'tenacity' or 'embrace change' or 'have clear goals'. Which are all good. But here is my answer:

- good idea - addresses a real existing need in a way no one else ever thought of in just that way

- hard work - you can't just sit back and let people recognize your brilliant idea. Hard work can be either doing more (of the same stuff) or doing different (thinking harder)

- strategic details - getting the nuances just right can make the difference, that one tiny little thing, the subtle finishing touch whose lack

- good team - well prepared, good skill coverage, good connections

- luck - there are some many things that you don't know and can't know, historical accident, the whims of people, unexpectedly meeting an old friend

Any one of these can ruin things, but also any one can put you over the top into success. Sometimes you -can- sit back and let it run if you get real lucky (this is -very- rare, you're not -that- lucky). Often trying harder can compensate for bad luck.

The usual question is which single one is the most important, a good idea or a good team, or personality vs substance. The answer is always yes, but one can often compensate for the other.

Which one is the most important? The point of question like this is to determine which single one can you put your efforts into. Obviously you shouldn't neglect any but some are difficult to change like luck, but that's why you need to be well-prepared for when the opportunity arrives (or the disaster occurs).

EBM, precision medicine: literal but more

Often the metaphorical meaning of a word is used more often than the literal one.

Medicine is a technical domain, but lots of terms in it are metaphorical.

EBM, or evidenced based medicine, is a label for a suggested desire to literally base medical practice on evidence rather than 'what you've always done'. Does extra sugar intake cause hyperactivity in children? It's pretty obvious it does. Except when studies are done, there is no appreciable difference in activity afterwards between children ingesting more sugar and those who don't.

We have preconceived plausible notions, but it's always good to check more scientifically. Isn't medicine always working on evidence, and equally dismissive of unscientific, non-evidenced based things like homeopathy, or ingrained myths like feed a cold starve a fever? Of course, but still there are reasonable plausible things that just may not have an actual effect. EBM came to refer to a trend in RCT, randomized controlled trials, which means a trend in a particular kind of government funding. Also it became associated with expensive measures to confirm really idiotically obvious things and parodied by the idea of an RCT for the efficacy of parachutes. So sometimes EBM sounds like a good thing, and sometimes it sounds like a dumb thing. But it mostly means 'be skeptical, do an RCT', whatever the nuances of funding and sample size are.

Personalized medicine, as a term, is also problematic. Literally it's saying medicine should be directed towards the individual differences. But that's so obvious, you're not going to treat someone for a broken left arm when it's their right that's broken. How people use the term 'personalized medicine' nowadays is for when gene variations are known about the patient. That's it. PM, the non-literal version, is for the handful of medical situations where the different gene variation (on a small set of genes) suggests a slightly different therapy. The expectation is that the science will expand to include lots of gene variations and problems. Yes, doctors have been using medicine personalized to a patient's family history, environment, social situation, problem itself (duh!), etc. forever. PM currently refers to doing that same thing but with some gene knowledge.

These two terms, EBM and personalized medicine, are not incorrect, but they have a much more specific meaning than you think if you've never heard them before. If you use them all the time, then you (implicitly or not) know their narrow usage. Luckily people whether they know it or not, don't use these in the broader situations.

SQL JOIN Venn diagrams are only sort of Venn diagrams

SQL is a standard for querying databases. Despite questionable pronouncements that SQL is Turing complete, I hesitate to call it a language because its power is in using boolean logic in dealing with tables of data whose columns point to each other.

And often Venn diagrams, the go-to visualization for set operations, are used to help explain the process of table JOINs.

The interesting things is that set operations and table joins are not really the same thing. They're related but just not the same. Set operations, which are pretty much the same as boolean/logical operations, are simple to visualize. The picture is the universe of elements, a circle surrounds a group (a set) of elements with a property, and a set operation does something to one or more sets to make a new set.

(from Modern Dilettante)

SQL also has set operations that combine tables as though they were sets: UNION, INTERSECTION, DIFFERENCE. They simply do the same as the set operations; two tables with identical column labels have their rows combined into a single new table (UNION means all rows in both, INTERSECTION where the column/row entries match in value, etc).

But this is not how Venn diagrams are usually presented to explain SQL. UNION, INTERSECTION, etc, are not the most useful of operations (the WHERE clause of a SELECT is where the booleans are most commonly used). Venn diagrams are most often used to explain JOINs. A SQL JOIN first matches on a field from one table and a field from another (presumably a field of the same type or kind).

(source Codeproject)

These Venn diagrams explain the difference between inner, outer, left and right joins perfectly...except they are just from a different world than the traditional set operations.  A JOIN is intended to merge the information appropriately in the n by m relation (where the size of A is n and size of B is m). The universe isn't the set of rows of both A and B together. The universe is the product of rows in both. And the difference between inner, outer, etc, is purely with how the JOIN deals with NULL/missing elements in A or B.

An INNER JOIN keeps rows of AxB only where both A and B rows exist. A LEFT JOIN is only when the A part exists (B may or may not), similarly for RIGHT JOIN. An OUTER JOIN doesn't care if either a corresponding A or B exists. So the boolean idea does apply but in a strange way, only with respect to the NULL condition of the matching field. If the value of the field from A has no matching value in the field for B, then B is NULL or missing then (and vice versa).

So the Venn diagrams for SQL operations, I can't really say they are true Venn diagrams; they don't show the state of a consistent property over all elements of the universe. Or rather the universe is a bit more complicated (depends on A and B, their cross product) and the property being booleanized is whether element of one table is NULL. You can't just take an arbitrary universe of elements (with properties. With JOINs, you have to create the universe, the product, first before examining the elements (and whether the A part or B part of the new row is null or not.

'Get in the groove': Something I assume nobody will bother implementing ever

You know how when you're driving a long the highway, and you get to a nice stretch where for minutes at a time there are no repairs or the slightest pot holes and you just drive along listening to the beat of the seams of the concrete slabs, and the grooves embedded in the concrete give a certain pleasant drone to the drive? And then you get off the highway and the grooves stop and life is somehow just that little bit more boring?

What makes that droning noise is the tires spinning along the pavement and the tone or pitch of the drone comes from the width of the grooves.

And that is the start of the idea.

The grooves are created at the time the concrete is laid  (this works for a concrete highly which will 'hold' the grooves, on a local street the asphalt is too soft to maintain the grooves). A different groove separation width (not the grooves themselves, but the distance between the grooves) would produce a different tone with the speeding tires.

So the suggestion is to modify the groove tracer mechanism, which scores the soft concrete being laid down linearly by the machine that lays the concrete, modify it so that the distance between tines is not fixed but is instead modifiable. Some sort of caliper action for the whole width of tines. And have this calibrated so that particular notes could be created in succession, allowing melodies to be embedded into the highway. Hey, go wild, you could have two part harmony if you groove differently the left half vs the right half of a lane.

So you could have say the highway leading into the airport playing the melody for 'Stairway to Heaven' or route 95 in northern NewJersey with the view of the Manhattan skyline playing 'New York, NewYork'.

Of course there are issues. Musically, the pitch is determined by groove width and to some extent by tire tread markings. The speed of the car doesn't alter the pitch considerably because the groove width is parallel to the tires' direction. The tempo would certainly change. There's the safety issue of the distraction of hearing this subtly melodic droning as you drive, but it's not as distracting as billboards. There should definitely be some signage alerting drivers to the sound so they don't think they're going crazy, can't get that tune out of my head.

I think this would only be realistically feasible for long stretches of road like a highway. If local streets were paved in such a manner, intersections and other slower cars would impede satisfactory 'playing' of the melody.

So now how to create a monetization model out of this. Commercial jingles, I-57 sponsored by Anhaeuser-Busch with the Budweiser theme? Christos superscale artwork?

Science is all mental

Science is all about figuring things out. How do I move this big rock? Which way do a turn the wheel when parked on a hill? Why is that guy such a jerk?

That's all dealing with real world things but we're doing it with our thoughts. The non-obviously obvious ways of doing it are:

Looking or using memory - You can't just make things up (which memory often does sometimes), so you have to look to to make sure you're not remembering wrong. Your mind may make ideas, but you should check them against reality to make sure you're not wrong/crazy.

Naming - we use language to communicate what we have ideas about with others. But frankly, just for ourselves, giving a name to something, using that name with something the same, giving another name to something that is different, those are all mental tools even for yourself.

Guessing well - names don't always fit perfectly or are vague, but start with one word and if that doesn't fit, then use another or create a new one.

Quantity vs quality - big litter vs few children - product strategy

There's a set of biological characteristics around number of children that seem correlated.

(picture source)

(picture source)

Some organisms, like mice, salmon, and dandelions, have lots of children in a litter. This is correlated with those children (or eggs or seeds) having little parental maintenance, small body size, short-lived as adults, not particularly robust, having a low probability of any individual one reaching maturity. Others have small litters or even just a single child, tend to large body size, long life expectancy, and lots of time and energy is spent in making it a high probability that the offspring lives to child bearing age.

The theory behind it is called r/K selection theory (I only described a vague phenomenon, not any reason or mechanism behind it). The r and K come form a simple equational relation those as probabilities to the number of offspring.

The theory says that the evolutionary pressure that's behind all these correlations is stability of the environment. 'r-species', those with quantity offspring like mice, tend to exist in unstable, unpredictable environments with a lot of room in their niche (high 'carrying capacity'). 'K-species', quality offspring like elephants, usually occur in stable (low population change) environments or close to filling the niche.

That said, there is a loose, not perfect, analogy with product design. Consider a widget making company. The company can spend all its energy creating lots of different widgets of acceptable quality. Some may work out some may not, but having a lot of then increases the chances that at least one will work out in the end.

Another company with the same resources may spend them all on making very few widgets but very refined specially made ones. Each of these few widgets are very likely to become successful because of the resources put into them

So, just like with animal reproduction frequency, it's a tradeoff. For constant resources, low probability survival of individuals can be offset by many individuals, and high probability of survival requiring lots of resources offset by having very few individuals.

For business, the corresponding explanation would be that in an unstable unknown business environment, having lots of products with slightly different features, can increase the chance of viability (eg web game companies have many games on offer but only one or two become famous). In a very stale business environment, with few companies selling few alternatives, a lot of company resources should be spent on just a few very large well-known products.

So this is just theory, and an analogy of a theory, and one can imagine many examples for which the explanation works, for which it practically works, and for what it just makes no sense.

'Agile' and 'lean' and 'fail quickly' are recent manufacturing trends. They are usually in contrast to the software development 'waterfall' planning method. Agile and waterfall don't necessarily apply well to the same kinds of software. Waterfall usually works best for a large scale well-understood design situation, and agile for a small project where all issues are not well-understood before hand and quick changes need to be made based on changing circumstances.

The analogy from biology to business to software development is not perfect but there are enough similarities to be recognizable.

One more analogy: scientific publishing. Historically (Europe), publishing of scientific knowledge started off as monographs (eg Aristotle, or at least that's all that has survived) and during the renaissance included personal correspondence which morphed into privately bound collections of papers and now (early 21st c) the library-industrial complex of journals. Textbooks and monographs are still considered the pinnacle, but are infinitesimally small by weight of paper, or more modernly, number of bytes.

Also, research has turned, during and after WWII, a government supported endeavor (where before only rich people had the means to do it).

There is so much research going on and papers being produced and science being fractured into narrower and narrower domains, that many papers are published (after being peer reviewed) and never read by anyone else. I'm not saying this is a bad thing. It sounds bad because that effort seems wasted. I'm just pointing out the analogy with quantity vs quality; the recent trend is towards quantity. Actually I don't think quality has necessarily bee traded off, just that the resources has exploded, allowing lots of failures to exist alongside really great advances that would never have come without the resources.

Comments on "10 things that sound entirely true but are false"

A recent lecture by Neil deGrasse Tyson is entitled

"10 Things we have heard and Re-told but are completely False"

Great lecture on mistakes in elementary astronomy  (only 2 slides for 10 minutes breaks the many slides rule in just the right way). You only need to know elementary physics and astronomy that you pick up in elementary school and life to understand how these are both right and wrong. Here is the main slide:

I want to comment on the flip between 'obviously right -> explanation -> so terribly wrong' presentation for each one, what the nature of the rightness and wrongness is. The generalities are that almost all of them are conceptual mistakes, but some are context mistakes and others naming mistakes. That is, some are actual errors in understanding, some are errors in misplacing the context intended, and others are mistakes in using the names themselves wrongly. No actual trick questions.

• What goes up must come down - this is a context dependent. In the context of a person throwing a ball in the air, yes, it must come down. In our limited experience that is entirely the case. But rockets are basically high powered throwing. And (most) satellites never come down
• The Sun is yellow - context. The little time we can actually look at the sun it is near the horizon when yellow is the predominant color. But most of the day it is up high and is blindingly white (it -could- be yellow but happens not to be).
• Weightless astronauts left Earth's gravity - with respect to say the ISS they loo weightless. But no they are falling around the Earth because of gravity along with the ISS. Also as far as words go, gravity goes forever, some satellites have 'left' = won't be pulled back in/not elliptical path.
• The North Star is the brightest - this is just a factual error, or rather an error of just assuming that an important thing excels in all aspects. Polaris is actually pretty faint in comparison to other main stars.
• On a dark night you can see millions of stars - an error of words (and counting?). there are a lot of stars you can see with the naked eye away from the city on a moonless night. But millions? If you count by the area of the sky, given average human eye acuity, there are 1000's. But millions is over reaching, just using the word 'millions' to mean 'a lot'. The Milky Way galaxy you say? Sure you can see the galaxy, but you're not actually able to pick out individual stars making up the milky band.
• Total solar eclipses are rare - This is a context error. Sure they're rare for your location, but not for the Earth. The band of darkness happens every couple years. Also, rare is relative.
• Days get longer in summer, shorter in winter - This is a naming error (also a little conceptual). Summer starts (by official name) on June 21st the solstice which is when the days start getting shorter. So if you include June in your meteorological summer, then yeah for part of a month the days do get a little longer before it goes backwards.
• At noon the Sun is directly overhead - conceptual: sure in them olden days, that was the definition of noon, when the Sun was at its highest (I'm not going to get into the complexities of directly overhead). Because of time zones and daylight savings, the clock time of 12pm has been set so that the Sun is mostly near the highest point, but further off depending how close you are to the border of a timezone. It's hard for the eye to tell how far off things are in the sky. 
• The Sun rises in East, sets in West - a naming problem, somewhat pedantic. Directly due East is a point on the horizon, and the sun only rises there on the equinoxes. Dates further away it is further away on the horizon (furthest at solstices). All it takes is looking at where the sun sets on June 21st. People just tend not to do that.
• The Moon only comes out at night - I think this fits all three. 'Comes out'? You only notice it at night. The moon is always there somewhere in the sky.  We've all seen a half moon during the day.

What are really the problems with EHRs

There are a lot of complaints about EHRs (2016). Too much useless typing, too many clicks to get what you want, records are not really available. scanned documents are a pain, release forms take forever.

The intended benefits of an EHR are obvious. Data gathered about a patient should be available to everybody who needs to see it, quickly and seamlessly, just like all the other rocket science apps that track our dating.

I see two major problems: data sharing, and user experience.

• Data sharing - electronic health records was never a community service. When a doc or medical situation of a small team needed an IT solution, it was solved only for that particular team or doc. Nothing was intended to be shared. This creates the data silos. It would be a perfect metaphor except real silos can exchange grain so easily just by trucking it over. There is also the other turn of phrase, standards, of which there are, comically, many. There's no universal heath record, or even univesal health patient identifier.
• User Experience, both data entry and retrieval. The pencil used to be the universal recording medium. It was infinitely creative, hobbled a bit by legibility. Typing is so ... easy... that you're expected to do it constantly, but you can't draw. For retrieval, the current EHRs have at best the most rudimentary search. The EHR for a single patient reads like an electronic phone book: if you know what you're looking for you can find it, but it doesn't tell you what the town is like. Everyone complains that you get a lot of data but you just don't get what is happening to the patient.

It's annoying to hear complaints with out solutions. For once I feel I have some.

• Data sharing - the world is going to have to spend some time and money making a universal health record, just like a utility. It's not difficult to do, it just takes some desire and money
• UX - there's a lot of deeply -thought out UX design that could happen. But really just a quick modification to the 'facesheet': add a 3 line text box for a few notes about current status. A lot more could be done but that would change things radically.

These problems are not rocket science. Technologically they are simple. Maybe labor is involved but not much thought.

Going away speech

People often ask you at the end (well, really a change) of a career if you have any regrets. Is there something you would do differently, now that you know the consequences, your deathbed confessions of life changing decisions, minor twists that inordinately changed the direction of later events, what you wish you had done or or what you wish you had not done or stopped doing over and over and over.

You will regret many things. But of all those individual things, you will regret the many times you've had to regret things and you will regret that you failed to regret the things you've done and haven't done. You will probably regret hearing this too.

But now that you are a half step out of our daily lives, I'll tell you this: at your new place, things will look up, things will look down and you may miss all those good ol' days and familiar likeable and happy faces, just feel comfortable in that, once the door has closed behind you, whatever happens back there, those that are left behind will end up blaming you for it.

How to sound native in a foreign language

How do you sound native in a foreign language. First, there's obviously the accent, how you get those weird vowels and inflection just right. There's just plain grammar, which words are masculine or feminine, is the the past perfect continuous participle or the simple hortative passive? Then there's just plain word choice: a sentence could be translated one-for-one but you just don't say it that way in the other language.

Most grammars/instruction of languages will give you this. They give you all the conjugations of all the regular and irregular verbs. They teach you the right preposition. And maybe you can give a lecture in high-speed particle physics or order cake at a coffee shop or even discuss basic politics with a taxi driver. And all this fluently.

But they don't tell you how to be influent like a native. How to make the mistakes a native speaker would make. Hemming and hawing and slurring and skipping unnecessary words and adding the slightest of hints at words that change the entire meaning of a sentence, all like a native.

So here's a list of things, some barely linguistic grunts, in English that you should learn in the other language of your choice to add that bit of informal fluency.

• Uh, um - /u/, /um/ just filler until you can think of the next word
• Hm - /h/ I'm thinking
• Hunh? (recently considered to be a language universal, the similar phonology) 
• What? (I didn't hear your or I didn't understand you)
• Uh hunh (yes) u hū
• Unh unh (no) ū ?ū
• Right? (wasn't I correct) also No?
• Hey ('watch out!' or 'look at me!')
• Pfft /f/ - expression of disdain
• Ha 
• just plain laughing - different in every language
• ow! (that hurts!)

Of course, these may not translate well, or translate at all. German has 'je', 'doch' etc, Chinese the sentence endings 'a', 'ba', 'ne'. What are the universals?

---

It goes without saying (but I'm actually saying it and that's contradictory. Does that go without saying?) that profanity and other taboos are a large part of sounding native. I hesitate to add that as another category of things to learn. First, because they are almost by definition part of an informal language that is not very public and so not that necessary for communication. And second, it's bad enough when a native uses it, but it's extra awful when someone with even the slightest hint of an accent mouths off; it's rude _and_ they didn't do it right!. That said, here goes:

• ow! (I hurt myself)
• dammit! (I made a mistake)
• Damn you! (you made a mistake)
• You bad person! (insults)
• Leave me alone
• You suboptimal person! ()
• Taboo body parts and functions (sex, death, family, religion, and excrement), stand alone or in combination.

I've gone almost beyond mincing, but I think these are universal situations for which you can give canonical examples and extrapolate from there. Some languages have their own idiomatic domains (e.g. Quebecois seems to only use taboo terms that are also perfectly fine vocabulary of the Catholic Church, Arabic seems to favor comparisons to animals). There is surely a lot of overlap in the list, and many possibilities for each one. Expanding to full phrases may involve all of them.

Deep Learning: Not as good, not as bad as you think.

Deep Learning is a new (let's say 1990, but common only since 2005) ML method for identification (categorization, function creation) used mostly in vision and NLP.

Deep Learning is a label given to traditional neural nets that have many more internal nodes than ever before, usually designed in layers to feed one set of learned 'features' into the next.

There's a lot of hype:

Deep Learning is a great new method that is very successful.

but

Deep Learning has been overhyped.

and even worse:

Deep Learning has Deep Flaws

but

(Deep Learning's deep flaws)'s deep flaws

Let's look at details.

Here's the topology of a vision deep learning net:

(from Eindhoven)

Yann LeCun
What's missing from deep learning?
1. Theory 2. Reasoning, structured prediction 3. Memory, short-term/working/episodic memory 4. Unsupervised learning that actually works

From all that, what is it? Is DL a unicorn that will solve all our ML needs? Or s DL an overhyped fraud?

With all such questions, the truth is somewhere between the two extremes, we just have to figure out which way it leans.

Yes, there is a lot of hype. It feels like whatever real world problem there is, world hunger, global warming, DL will solve it. That's just not the case. DL's are a predictive model machine, very good at learning a function (with lots of training data). The function may be yes or no, or even a continuous function, but still it's take an input and give an output that's likely to be right or close to right. Not all real world problems fit that (parts of them surely do, but that's not 'solving' the real world problem.

Also, DL's take a lot of tweaking and babysitting. There are lots of parameters (number of nodes, topology of layers, learning methods, special gimmicks like autoencoding, convolution, LSTM, etc etc with lots of their own params). And there are lots of engineering methods that have made DLs successful, but these methods aren't specific to DL. Lots of better data, better software environments, super fast computing environments, etc etc.

However, there are few methods nowadays that are as successful across broad applications as DL. They really are very successful at what they do and I expect lots of applications to be improved considerably with a DL.

Also, for all the tweaking and engineering that needs to be done (as oppose to the comparatively out of the box implementations of regression, SVMs and random trees), there are all sorts of tools publicly available to make that tweaking much easier: Caffe, Theano libraries like Keras or Lasagne, Torch,  Nervana’s Neon, CGT, or Mocha in Julia.


So there are lots of problems with DLs. But they're the best we have right now and do stunningly well.

Kinds of Data: there are more than just the basic four

The science of statistics specifies that data points come from four basic types:

• nominal - these are incomparable labels like truth (yes, no), color (red, blue green), country (UK, France Germany, Italy). There is no relation among these elements there other than that they are in the same set. All you know about them is their names and that a name is different or the same as another.
• ordinal - only a rank is known (1st, 2nd, 3rd...) and nothing else (we don't know how far ahead 1st is from 2nd), just the order, like finishing order in a race.
• interval - we know the distance between any two elements A - B  like the height.
• ratio - we also know the ratio of two numbers where for example A can be twice B, like half-life of an element.

Notice how I describe these both mathematically and conceptually, because often a set selected from a mathematical domain, like the reals, can be interpreted in any one of these. For example, from the reals, they obviously have a ratio by division, and a distance by difference, can be ordered by 'less than', and can be categorical by using cutoffs, say >= 0 for yes and < 0 for no.

Of course, as with most systematizations, this list came after years of using methods that were created to work with whatever data was at hand, and then when the data just didn't work with those methods, new analogous method were created, or entirely new methods created for quite different purposes.

Statistical procedures seem geared to work with one of these types. Chi-squared on contingency tables are good for categorical data. Wilcoxon signed ranks for ordinals, t-tests for integral data, Poisson for count data. But mostly there are just two kinds discrete and continuous which fall to nominal/categorical statistics and pretty much all the rest of statistics respectively.

Existing science isn't as deliberate as a current systematization, as monday-morning quarterbacking/textbook-writing may make it seem. It's more incremental, and filling in gaps as needed rather than laying out the system ahead of time. You have a problem and you use a tool that works good enough right now, you develop that tool incrementally until it metastasizes well beyond it's initial conception. Contingency tables are great ways of summarizing tabular data, but you may want to do a significance test like all the t-test guys. 

---

Any kind of systematization is an oversimplification, forgetting possibly irrelevant details to make different things look alike, and placing a particular item into that systematization is also forgetting possibly irrelevant details to make it look like one of a few categories. But sometimes those details are not so irrelevant.

Binary data is a subset of nominal data, with just two categories. Two by two contingency tables and logistic regression are especially designed to deal with them.  Some multinomial categories will have some minimal relationship, say geographic location with countries, or wavelength for colors (colors are very complex because the brain processes them by multiple systems involving the wavelength, opponent process pairs, or beyond. Rank data is ordinal by definition, but when encoded as numbers, can be processed as interval or even ratio data (depending on the interpretation desired.

These four data types work very well for statistics. But it seems underspecified. We're used to measuring quantities or counting objects so all those categorical and interval methods apply so well. But there's so much more structure to the way things can be measured. Not humanities-style vague, wordy, qualitative description. Perfectly exact, just not necessarily a number.

There is an existing method for description of data. A very rich description method. It's mathematical notation. If data should be treated continuously, use R. If a vector over integers, Z^n. If an ordinal set, then that's a total order. If categorical, then you have a simple set. If the elements are related to each other one on one but in a complex restricted manner, then maybe a graph is the way to notate things. if the elements allow certain operations but not others, then maybe it's from a particular algebra, a Hilbert algebra, or instead a Banach algebra. 

Measurement is not always in the elementary numbers we count or measure or weigh with. There can be quite a bit more structure in the measurements than just a number.

There are no exact synonyms

There are no exact synonyms.

That may sound a little extreme, especially given that thesauruses exist.

There is no pair of words where one can replace the other in all circumstances.

'Bail out the canoe with a bucket'

Can you replace 'bucket' with 'pail'. Of course. But can you say 'kick the pail'? No, of course not, that would be wrong. You can't always replace a word with its purported synonym.

Well, OK, there are some circumstances where there are exact synonyms. In technical circles, especially the sciences and math, there is a special way of attaching a word to a definition. In technical areas one 'stipulates' a definition of a word. That is, you give a word a definition that is simply a shorthand for replacing a word with its definition. You're stating authoritatively that for a word, you must treat it like an exact replacement. Often these technical terms are supposed to be evocative or metaphorical, supposed to give you a good idea of the intended meaning. But you can have whatever mental connotations that help you remember the true meaning but the true meaning is what has been stipulated, it doesn't matter, A=B and that's all there is no more no less.

But with non-technical words, there is no stipulation. A word is just a trigger for some associations. And if it sounds different, then there is no way it can be identical in all situations. Different stimuli can give different responses.

I will even go so far as to say that even a given word is often not its own synonym because all words have multiple meanings.  I'm not even talking about homophones (words that are spelled differently, but sound the same, like 'horse' and 'hoarse') or the other side homographs (words that are spelled the same but can have different pronunciations and meanings, like 'bow' a knot in a ribbon or tie, and 'bow' the front of a ship). I mean a word that is spelled and pronounced the same but has a different but related meaning. For example, 'run' is a verb to move fast by your legs, but is also a noun for a  long rip in a stocking or a small stream.

The point is that if you desire a synonym, you can get that from a thesaurus, but it may not slot in perfectly as a replacement. And even a single word may have many associations and alternate meanings that it is not good in that slot itself.

Free will?: Science assumes determinism

Whatever the philosophical decisions made about free will versus determinism, science (or factual knowledge) attempts to discover everything that is deterministic, and to that end almost assumes determinism.

The only thing counter to this presumed determinism is its literal negation, non-determinism, which is modeled using probability. And probability is just shorthand for what we don't know or can control yet. This applies all the way from physics to sociology.

What's the point in these theorems?

Sometimes math is weird. Often you know exactly why a particular math thing is interesting. Like it's so obvious that algebraic geometry is there to help figure out where really weird multinomials intersect. But other times, even for simple things for which there's lots of research and historical precedence for concern, I just don't get it. Here's a list of things I just don't get. I don't understand the point of pursuing them. I understand the mathematical process, I just don't get the point:

• Craig's Interpolation Theorem in logic, if a implies c, then there exists b such that a implies b and b implies c and b only involves the intersection of vars from a and c
• Curry's paradox and Löb's theorem - I have trouble following the elementary proofs of these. They seem to say you can prove anything "'if X is the case then Santa Claus exists' proves Santa Claus exists' or something
• Herbrand's theorem - proves universals using examples?
• the Deduction theorem - it just seems so obvious. It's just Modus Ponens, right?
• quadratic reciprocity - allows computation of square roots in modulo arithmetic. Why you would want to do that, I don't know

I want to understand these things, and I can (usually) follow step by step manipulations, but I just don't get what they are for and what the point is.

Two-pass forward-backward approximation systems

A number of approximation methods work in a two stage cycle, a forward pass to compute the test on the system, and then a backward pass to update the system with the error of the test with respect to the supervised true answer.

In the backward pass, un update function moves in the opposite direction of the edges, update weights as it goes a long.

Neural Networks/ Deep Learning - a directed (usually acyclic) graph with weighted edges used to compute a function. In the forward pass, the values at any node are computed as the dot product of the value at the source nodes and the edge weights, then a simple threshold function (in topological sort order). starting from the input nodes (no in-edges) This computes the values at the output nodes (no out-edges). Traditional neural nets have a layer of input nodes, a single layer of hidden (interior) nodes, and a layer of output nodes with no edges directly from input to output. Deep neural nets have more layers. Arbitrary undesigned graphs are not usually not very successful.

Expectation-Maximization approximation of parameters of a statistical model. First the expected value of the likelihood function is calculated, then the model parameters are calculated to maximize that function

Primal Dual linear programming for maximizing a target function restricted by a set of linear constraints. The difficulty is dealing with the possibly large set of constraints and large set of dimensions.

Kalman filter successive measurement refinement. This is usually applied to position measurement with slightly fallible sensors. From an initial (fuzzy) position and direction of an object at time t, the position/direction at time t+1 is predicted y combining the prediction of movement from time t plus a fuzzy sensing at time t+1. Combined, the variance is lessened.

Are any of these even more alike than just the forward-backward pattern? Are there any other algorithms that are superficially similar, have a two-step iterative process?

Update (5/24/2016): this blog post seems to give pointers to how backprop, primal-dual LP, and Kalman filters are interderivable

My pet language peeves/non-peeves

It really bugs me when other people say (my inner prescriptivist):

• for 'often, pronounced 'off ten' instead of the correct 'off en'
• 'Between you and I' instead of the correct 'between you and me'
• pronouncing 'forward' as 'foh ward' (no first 'r')
• 'comparable' pronounced 'com `pair able' not ' `com pruh ble'
• Dwarfs roofs baθs instead of dwarves, rooves, baðz
• pronouncing 'processes' as 'prah cess eez' instead of the correct 'prah cess ehz'
• 'my bad' and 'back in the day'
• whilst/amongst
• Nutella as New-tella. I prefer  Nuh-tella

Conflicted

• 'irregardless' - obviously a sign of not caring about words, but it still takes me a second to register it as 'wrong'

It really doesn't bother me at all to say:

• 'Hopefully' to modify a sentence
• Feb you ary

I only recently learned how t pronounce correctly:

• awry. I used to say 'aw ree instead of uh 'wry

Wow, is that it? I was sure these lists would be a lot longer.

How hard is it to learn a foreign language?

How hard is it to learn a foreign language?

If you're like me, you grew up speaking a language. Since you're reading this in English, that was either your native tongue, or you've learned it at school. (If you don't understand this then thang nabbit presumably on the slindy intercongruenda, but that's neither here nor there)

But in the current cultural situation, native speakers of English (NSE) are at an intellectual advantage over speakers because it is the preferred language of diplomacy, science, popular culture. I could make a case that this is a disadvantage for NSEs but for the moment let's just assume that an English speaker is about to learn another language.

Learning a language involves a number of separate domains which are somewhat independent. There's:

• grammar (syntax), how words are ordered and modified to tell you their function with relation to the other words in a sentence
• vocabulary and sayings, the dictionary of words, what you call that thing or action
• pronunciation, the accent, those weird sounds that make up the words. 
• writing and spelling, how the language goes on paper or screen. Though writing is not really language itself (language was spoken first and then individuals created writing after the fact), it is often the easiest entry into language learning and offers higher volume of consumption per unit time. Non-roman alphabets or complicated spelling rules make it difficult to learn how to pronounce things from writing (certainly for beginners).
• culture, the supporting tools in your environment that ease language learning, books, media, acceptance. Not traditionally thought of as a part of learning a language, but makes a huge difference in learnability. The internet is making a lot of things just universally available: news and movies, ordering books.

Here is my assessment of how difficult it is for NSEs to learn very specific foreign languages, (in misleadingly quantifiable but semi-educated subjective terms: 1 very easy (little to no effort) , 2 easy, 3 medium, 4 hard, 5 hardest) As usual it is easy to disagree after the fact (after you've spent time learning), but these are what I think the effort will be for those who don't know yet, within 1. Also, 'easy' means easy with respect to many languages. You'll probably still struggle with even the cognate words at first. But some languages don't have even that to fall back on. Also, there are two kinds of difficulty. One is that there is a new distinction, a totally new sound that is hard to do or remember, the other is the scale, usually just a number of things to remember. So there might be many new one-off distinctions (a new 'r'), or there might be a single distinction with lots of instances (like gender on nouns), or both (Russian verbs).

So here is a structured list of those issues for each language/language group.

---

French - pretty easy overall. French (and Latin cognates) form most of the educated language.

• grammar - 3 conjugation of verbs in a few tenses but very regular, gender of nouns to remember. Word order mostly like English
• vocabulary - 2 once past the basic words, educated vocabulary is almost identical. Very easy to guess meaning.
• pronunciation - 2 mostly the same as English, no distinctions but a few strange sounds to get the accent right (u, nasals)
• writing/spelling - 2 Latin alphabet, spelling very regular, a few accent marks to be aware of, a few unspoken letters at the ends of words. 
• cultural - 1 taught by default in the US/England. In the US, products (few of them) that have bilingual packaging are in French for Fr Canada). Easy to get French things to read for learning.

---

Spanish/Romance - pretty much the same as French except slightly easier, because of close cultural connections (in US) and simple grammar

• grammar - 2.5 conjugation of verbs but very very regular, gender of nouns to remember, verb conjugations but that's it. ser/estar, por/para only difficulties
• vocabulary - 2 once past the elementary words, educated vocabulary is almost identical, part of the European group, easy to guess meaning.
• pronunciation - 2 mostly the same as English, no foreign distinctions but a few strange sounds to get the accent right (g, j, h, r)
• writing/spelling - 2 Latin alphabet, spelling very regular, a few very regular differences. Very wysiwyg
• cultural - 1 large subpopulation of Spanish speakers. Currently the default foreign lang taught in  in the US. Lots of media that is Sp available.

---

Latin - The grammar is a lot: conjugations, declensions, gender, agreement. But the vocabulary is very recognizable. Or if it is not recognizable at first, then there will probably be some mdeical, legal, erudite term in English that corresponds.

• grammar - 5 conjugation of verbs but very very regular, gender of nouns to remember
• vocabulary - 2 once past the elementary words, educated vocabulary is mostly English looking. Half of English tech vocabulary is Latin or Latin-derived French.
• pronunciation - 2 since it's mostly used literarily, pronunciation doesn't seem to matter much. But if you have to, it's very organized. When in doubt sort of like modern Italian. 
• writing/spelling - 1 Roman alphabet (the original), spelling very regular, a few very regular differences.
• cultural - 2 The source of European culture. English law and medical vocabulary. Lots of erudite literature. Very few children's stories ('Winnie Ille Pu' is about it) and no news. Even though it is not used in the real world as such, makes law, medicine, and all romance languages very easy to learn.

---

German - you think it is going to be easy because English and German have close historical roots, but it's not that easy.

• grammar - 3.5 conjugation of verbs sounds like English, gender and 4 cases of nouns to remember, word order/prepositions just slightly different enough to English to be annoying. 
• vocabulary -  3 a lot of similarity with elementary words, educated vocabulary is loan translations from Latin (that is, not Latinate, but the pieces translated to German pieces) so visually don't look right, but are made up of squashing basic words together. Vacuum cleaner = Staubsauger = dust sucker. Easy to guess.
• pronunciation - 2 mostly the same as English, no distinctions but a few strange sounds to get the accent right (r, ch, pf, v/w/f)
• writing/spelling - 1.5 Latin alphabet, spelling very regular
• cultural - 2 not part of US/English linguistic consciousness. Not taught at schools. Large subpopulation of people with German heritage (both US and England) that is entirely ignored as a heritage. Yiddish, a variant of HG, has a larger cultural heritage (from a much smaller population). But some things are in the fabric of English culture (Grimm's Fairy Tales, general northern European culture)

Russian - for IE, hard to learn basics, hard to master because of so many rules.

• grammar - 5 conjugation of verbs sounds like English, gender and 7 cases of nouns. And even more exceptions to these. Getting the basics is hard and then it steadily gets worse. 
• vocabulary - 4 there is a lot of borrowing for educated vocab directly from English, but no where near as much as English from Latin. So most vocab unfamiliar
• pronunciation - 3 lots of consonant clusters and weirdly placed palatalization, and 
• writing/spelling - 3 Cyrillic alphabet, spelling mostly regular, except for 'o'
• cultural - 3 not part of US/English linguistic consciousness. Not taught at schools. Small subpopulation of people with German heritage (both US and England) that is entirely ignored as a heritage.

Irish - This is the most surprisingly exotic. It is definitely Indo-European, but it changes as many parameters as possible within that system to make it just plain weird. Nothing seems to be regular at first sight. Also at second sight. But eventually it is very regular, just there are so many context rules that are hidden or look like they overlap, but actually aren't

• grammar - 4 if only gender were the only difficulty, but there's so much more. VSO, prepositions that seem to conjugate like verbs, lots of initial sounds changes with exceptions on exceptions. The rules are regular but getting them is hard for a beginner. Oh, and 5 declensions for 2 cases with nouns.
• vocabulary - 3 not really cognate, except maybe numbers (but to complicate it, two sets of numbers that lenit or elide depending on the number). Plurals and verb roots don't seem regular at all.
• pronunciation - 3 the sounds aren't strange (except for 'ch' and 'dl' and 'gh' and dh' and slender vs broad and etc etc), it's the sound changes based on grammar that are insane.
• writing/spelling - 3.5 Latin alphabet, spelling is very regular but in the most misleading way. Lots of letters (vowels and consonants) sound nearby but different enough their English counterparts, if they're sounded at all. Should be 4,  except it really is rule based.
• cultural - 3 Taught in Ireland, not in US. Very small world community. Huge subpopulation of people with Irish heritage in all former English colonies...where Irish is entirely ignored as a language heritage. Not much out there to read.

---

Persian - This is the most surprisingly non-exotic. No gender, very little conjugation of verbs straightforward word order.

• grammar - 2 no gender, little conjugation of verbs
• vocabulary - 3.5 some cognate words, but educated has a lot of Arabic borrowings. But mostly unfamiliar.
• pronunciation - 2 a couple of uvular throaty g's (gh and kh) but not the sore-throat of Dutch or Arabic
• writing/spelling - 3 Arabic alphabet, no vowels. Hard to get pronunciation unless heard before. Also it seems like all typesets of Persian/Arabic are much much smaller for a given fontsize
• cultural - 4 very little connection with the West. Mostly Islamic. 

---

Arabic/Hebrew - exotic grammar and writing, but at least it has an alphabet.

• grammar - 4 conjugation of verbs and nouns similar not by endings but by vowel changes within the root, gender different enough to English to be annoying. 
• vocabulary - 4 entirely different. some direct English borrowings but not enough to make a difference.
• pronunciation - 3.5 a few extra throaty sounds and s/t/th's that they distinguish but don't exist in English
• writing/spelling - 3 Semitic alphabet (no vowels) (Arabic and Hebrew are similar like Roman and Greek), so you have to know things first to read properly (makes learning difficult)
• cultural - 3 Hebrew has a lot of cultural connections (Christian and Judaic vocab). Arabic has the difficulty that MSA (Modern Standard Arabic) is not really used in the street or at home, and each country has its own dialect (which two countries over may not be intelligible). Why Arabic and Hebrew together? Salaam/Shalom should be enough to convince you

---

Chinese - exotic, learning writing takes as much work as an additional language. I'm referring to Mandarin, not Cantonese or Shanghainese, which are unintelligible foreign languages (very similar but unintelligible as English is to German). But just as hard (or harder if you think having many more tones is harder. which it is).

• grammar - 2 no gender, no conjugating, no declining, no nothing. Straightforward order (slightly different from English), few exceptions. 
• vocabulary - 4 entirely different. But like German, new words are created from pieces of smaller words. Semantically it's not always obvious how though. Words and phrases made from these short simple VC words have this otherworldly feeling that you can't get a hold of them, maybe because they're shorter or all sound a like.
• pronunciation - 3 tones are exotic, and distinguishing some palatals is exotic. but that's it. lots of near homophones (see pronunciation) so lots of room for misunderstanding or puns. Not as bad as it seems. 4 for tones and palatals but 2 for everything else. Cantonese has like 12 tones, so count yourself lucky.
• writing/spelling - 5 Ideographs, every syllable has a picture. Like learning an entirely additional language. It's not as terrible as that; most characters actually have two distinguishable parts, one as a hint to pronunciation and one. Supposedly only 2000 characters needed to read a newspaper. Only?
• cultural - 4 very different cultural history so interpersonal expectations can be very different

Japanese - exotic, shares ideographs with Chinese but the spoken language is entirely different

• grammar - 4 (my vague impression)
• vocabulary - 5 entirely different. (my vague impression)
• pronunciation - 4  (my vague impression)
• writing/spelling - 5+ Chinese borrowed ideographs, plus two syllabaries (hiragana, katakana), and oh plus maybe another alphabet. So like Chinese but worse.
• cultural - 5 very different cultural history (my vague impression)

One take away is this, that because of extensive borrowing from nearby or imperial cultures, languages that are different grammatically may have learning eased by having some vocabulary overlap. So European languages tend to be easy to learn among each other for pronunciation and vocabulary (and if one Romance is your native language then all but Latin are all 1's), Arabic and Persian with vocab, and Chinese and Japanese with respect to writing.

Of course this is an entire oversimplification.  There are much more refined metrics on language learning: a scales of language proficiency like ILR (0 to 5), CEFR (A1 to C2), or ACTFL (novice to distinguished). And each level has published expected hours of instruction to reach it. So these are admittedly very subjective assessments on my part.

The best way to learn a language is to be born into it. If you can't do that, have relatives in your household who speak it and nothing else so you can't rely on the one you already know. As those strategies are not always at hand, immersion is the next best thing (but also where you can't fall back on the crutch of your native language). As that often poses a chicken and egg problem, your most realistic first attack is study habits, lots of listening and reading. And the latter is what I think I am 'scoring' for English speakers.

It's only fair to go the other direction. One might think that it is a simple inverse of the above (or really since difficulty is a distance, exactly the same numbers as above. But that's not exactly the case. There are language to language difficulties (or simplicities) and universal ones. I will oversimplify considerably and talk about all languages trying to learn English.

• grammar - 2 basic grammar as simple as can be. No gender, little inflection or irregularity. prepositions and phrasal verbs have lots of exceptions and ambiguity. I'd give it a 1, but I'm trying to account for familiarity bias.
• vocabulary - 2 easy for Europeans, not so much for outside of Europe. 
• pronunciation - 3 should be given 5 for 'th' alone which hardly any other language in the world has. Short i and diphthongs are weird but you'll get over it.
• writing/spelling - 4 For a logical roman alphabet, its spelling is atrocious. There seem to be more exceptions to exceptions than the rules themselves. Almost like ideographs, you feel like every word has to be learned by itself.
• cultural - 1 Because of English colonialism, American economic strength, entertainment media, and commercialization, English is everywhere. Movies, music, TV, the internet, everything you buy, something about it is going to be English. Everything is available everywhere in English. Even if it is exotic to you, it is so in your face you can't help but know about it. People elsewhere know more about English speakers (mainly Americans) more than they do themselves. There's so much opportunity for learning English.

If it weren't for the spelling part, English would be ideal for learning (only Chinese and Japanese are more difficult to learn in orthography). Grammar, vocab, pronunciation are easy for everybody.

Sorry if I left yours out. I'd like to add in other representatives, Slavic (Czech), sub-saharan African (Swahili), Indian subcontinent (Hindi and Mayalayam) and southeast Asia (Thai and Indonesian), to get a better world coverage.


Handy table to compare (and more easily see where you disagree with me)

Language	Grammar	Vocabulary	Pronunciation	Writing/Spelling	Culture
French	3	2	2	2	1
Spanish	2.5	2	2	2	1
Latin	5	2	2	1	2
German	3.5	3	2	2	2
Russian	5	4	3	3	3
Irish	4	3	3	3	3
Persian	2	3	2	3	4
Hebrew/Arabic	4	4	3	3	4
Chinese	2	5	4	5	5
Japanese	4	5	4	5.5	5
English	2	2	3	4	1

Weird pronunciations in medicine

- pathology - obviously pronounced puh-thology (in IPA /pə 'θɑ lə dʒij/) But all I ever hear when docs say it is path-ology (IPA /'pæθ 'ɑ lə dʒij/), the first syllable not unstressed sounding like 'path'. Since everything in life must have a reason, I wonder what it is. Is it an attempt to differentiate it from something that sounds similar? Do they just want to emphasize it somehow?

- patent - obviously pronounced pa-tent (in IPA /'pæ tent/). Like patent attorney, patent leather shoes, patently false. But docs use it to refer to, say, a vessel or duct or that is not collapsed, that is full of liquid or air keeping it mostly cylindrical-ish, never mushed or squeezed down (blocked or occluded or limp). And when they do so, they say pay-tent (IPA /'pej tent/)

There are others. Doctors are weird.

Foundations of Math: Sets vs Categories vs HoTT

There's been a controversy lately about the foundations of mathematics. It was controversial when it first was discovered in the late 19th/early 20th century when the foundations were being developed, controversial mostly because most mathematicians didn't feel its necessity and also because as a young field things just hadn't been resolved. But some of the greats (Hilbert, logicians) thought foundations were needed.

The development of foundations started almost entirely with logicians and philosophers (Boole, Cantor, Peirce, Frege, Zermelo, Russell) creating the axioms of set theory all based on the concept of set membership. This was not particularly in the center of mathematical development. It was essentially completed in the 1920's, then controversialized by Gödel in the 1930's, but was refined by him and other logicians (Church, Tarski, Putnam,) and set theorists over the next 30 years.

Then in the 60s, an alternative foundations was proposed and that was category theory all based non the concept of maps betwen objects (morphisms) representing similarity. It felt like category theory could model most any mathematical field that way, even if objects and morphisms were not central, and the diagrams it created seemed to show similarities among the disparate fields of math. Category theory itself was created in the 40s by MacLane and Eilenberg but Lawvere began in the 60s to think of it additionally as an alternative foundations.

More recently, the 2000's, higher order type theory (HoTT) has been proposed and developed as an alternative foundations (mostly by Voevodsky). I don't know much about HoTT other than it seems like a subset of Category Theory.

Without going into the details of each of these foundations, it seems to me, as a semi-educated outside who knows these fields very superficially, that the word 'foundations' means something different to these different alternatives.

For set theory, it is a foundation for reasoning, for specifying proofs, for how you can justify things. For the others, the foundation is for content and similarity among branches of mathematics, what is known in mathematics. That is, these aren't really exclusive foundations, they both have their purposes and they are non-conflicting they're just different kinds of formalisms rather than competing for the same ideas. It is also an intra-mathematical result that they are interpretable in each other; the concepts in one are translatable to concepts in the other. You can do HoTT with set theory as the underlying basis and similarly you can do set theory with elements defined in terms of HoTT.

My preference is for set theory. Another reflection on this is that set theory and axiomatics is easy to explain early (to those with less mathematical maturity) using very basic mathematical analogies, and category theory and HoTT is more difficult needing lots of higher math to understand and use as non-trivial examples. Also, set theory includes logic (somehow, I'm not sure if is as additional machinery or if it is considered a consequence of set theory), and I don't know what kind of inference mechanism is allowed with HoTT or category theory.

Of course this is entirely biased of me because I think that Turing Machines form the best foundation because it is very simple to understand and is entirely operational, and is a metaphor for all computation which is all that mathematics is good for anyway!

Can Deep Learning be applied to Automated Deduction?

Deep learning is just a deeply layered neural network, and by deep they mean more than one internal layer. It has recently gained much attention because of its successes with images and go and speech and all sorts of things plain old NNs weren't doing so well at.

But what about automated deduction? That's not rhetorical, this is entirely speculative and answer free. I am wondering if DL (or any ML technique) could be thrown at AD (or ATP (automated theorem Proving) whatever you'd like to call it).

First the optimism: wow wouldn't that be cool, a method that would prove really hard mathematical theorems (the ATP community has to work hard and do a lot by hand to do things like flyspeck) or even . But desiring the outcome doesn't say anything about the implementation.

Next the pessimism. By analogy, images and speech/text are iffy optimization problems. Lots of perturbations get you the same thing. But logic (and similar combinatorial problems) are all or nothing. It is either true/proven or false/wrong. how could an approximation optimization method translate to logical combinatorial problems? Where do you get the scads of supervised instances required by DL? Millions of tagged images from MNIST but from TPTP literally only thousands.

So, I don't know. But just because I don't know how to do it doesn't mean it can't be done.

A small nitpick about a small p-value problem trope

Lately the p-value has been getting a lot of press, almost entirely bad (tip of the iceberg). Whatever it is and means has been up for discussion as it hasn't had since NPHT was created (the Bayesians have been fighting against it, or rather fighting for alternatives, since the 60's).

Hidden in the middle of this storm, or a small tornado on the side, is the issue of data fishing or p-hacking. Since only a p-value of less than .05 is considered 'statistically significant, only such values are considered publishable leading to the problems of: selective publishing (ignoring significant non-results), and p-hacking (if one p-value isn't good enough, change you hypothesis and testing little by little until you get a p-value below the threshold). The problematic trope is stated in roughly the following manner:

At p-level threshold set at 5% (or .05 or 1/20), all you need is 20 studied hypotheses to get one hypothesis that is significant by chance.

It's so obvious!  With 1/20th probability, you need 20 tries to guarantee a hit! The intention is that you shouldn't make many hypothesis tests at a time, otherwise you'll get some false hypothesis stated as true.

But you may notice with that wording that it is a classic gambler's misinterpretation, 'the run has to end!'. Each hypothesis test is independent, and so the probability of the next test will not change if all the previous tests are all hits or all not or whatever.

Whatever you think of p-values, and whatever you think they mean, they are probabilities. Probabilities of what is complicated and nuanced and misleadingly stated and problematic and the firm basis for statistical inference for the past hundred years. But still, they are probabilities of something and a strict threshold of 5% of accepting a hypothesis over rej... forget that verbiage. it's a 1/20 probability event.


So now we're in the realm of basic probability (and its own difficulties) but they should be shared by Bayesians, Frequentists, Kolmogorov..ans, Keynesians (he had his own!). So any hypothesis has a probability of .05 of being positive, a hit. What's the probability of a hit in 1 trial? 5%. What's the probability of a hit (at least one hit) in 2 trials? 3? n trials? Those are harder but only a tiny bit, basic probability/combinatorics. What's the probability of at least one hit? One minus the probability of no hits at all. What's the probability of no hits in n trials? (probability of no hit in one trial)^n. They are independent events so you multiply. Final answer:


\[
P({\rm hit\ in\ n\ trials}) = 1-P({\rm no\ hit})^n = 1- 0.95^n
\]

Well, not final exactly, it's just a formula. We don't yet have a good picture of what it means in realation to our intuition about 'it'll take 20 to make sure we have a hit'.

So then a picture:



It starts at 0, rises in exponential decay asymptotically to 1, but is a little slower than you'd expect for an exponential because the base is so close to 1.


The usual way to present such probabilities is, like the birthday paradox, to say how many trials it takes to get %50 chance. Multiplying .95 a few times we see that it takes 14 trials for there to be more than 50/50 chance that at least one item is a hit. To get to 95% chance, it takes 59 trials. There's no guarantee that there'll be a hit, just the probability gets smaller and smaller.


I make this point to... well, to pick a nit. You do more experiments, the more likely there will be one that is 'statistically significant' totally by chance. Intuition and logic lead to that immediately. But the logic is never done and one step of it doesn't lead to correct two steps. I do realize it is a bit of a mouthful and difficult to digest to say 'after 14 trials there will be a fifty-fifty for a false positive'? 50/50? It's not obvious how that relates to 5%, but the erroneous 20*.05 = 100% does obviously relate.


In the end, to say '5% means 20 experiments', which seems so directly and intuitively obvious, is wrong. In the right direction, but wrong.