The poor man's math blog

Some photos and statistics

Fri, 12 Jan 2018 00:00:00 +0000

I recently posted 120 of what I considered the best photos I took on Facebook. Here’s some stats about how much my friends liked them.

Ideally I posted one photo per day, named from 1 to 120; but life happened and I had a hiatus of a few weeks. Regardless, let’s assume that each day there was a photo, so the index is also a time quantity.

We can start with some high level stats: there were 867 reactions, that is a little bit more than 7 per photo. While posting, I got the impression the novelty wore off after a while, and friends started to get tired. Indeed, a simple linear regression shows a negative coefficient.

Reactions per photo

Photos with certain number of reactions

If we group by number of reactions, we the histogram on the right.

It looks like a decent Poisson distribution. Indeed, we can assume reactions happen only on the day each photo was posted, and so the number of reactions per day should indeed be a Poisson. Of course, this also assumes reactions are independent on the quality of the photo… maybe that’s a good assumption.

The top photo has 25 reactions, and other 7 have 15 or more reactions. Here they are, in order (left to right, top to bottom):

Photos with the most reactions

Speaking of low-lights instead, here are the 5 photos with only one reactions (we will soon talk about that):

Photos with only one reaction

Overall, I think it’s fair judgement. There are some weirdness: the top right best photo (Hallgrimskirkja, in Reykjavik) has probably many laughing reactions due to the witty pun I added as comment, rather than due to the photo itself. On the other hand, I would have expected the first worst photo to be appreciated more.

So, why no photo has no reactions? Well, let’s see a histogram of how many people liked a certain number of photos:

People with certain number of reactions

You can recognize a pretty good power law (alpha = 1.8), but also some outlier; the person that liked 112 photos (93%) as you can imagine, is my mum. The most likely explanation for the remaining 8 is some bug in Facebook’s news feed.

The reason for this silly post was just to try out seaborn, a plotting library for Python on top of matplotlib. It makes easy to plot a limited set of graph types, and in addition has a decent preset style: that is to say, exactly the opposite of matplotlib. So I couldn’t really finish without using the coolest graph type it allows: violin plots.

I split the 120 photos in 9 categories, here listed from the most frequent to the least: architecture, cityscapes, landmarks, landscapes, people, close-ups, animals, nature, and textures (landscapes and nature are similar, but the former has some human-sourced component). And here’s the reactions in this sets behaved:

Reactions per tag

Beautiful. Animals and nature are the overall favourites, which again shows: the Internet loves its kittens.

Filling a void

Mon, 11 May 2015 00:00:00 +0000

In the increasingly populated scene of London mapping there was still something very basic missing.

London boroughs constitutes an important part of the administration of the city: they are responsible for many local services: school, social services (including housing), waste management, parks, and have a say also in the public transport. To fund them, resident pay a council tax which varies a lot between boroughs. Also, the elections for the borough council are the only one where non-citizen can vote, given their local scope.

London borough map on Wikipedia

One problem is that as of yesterday I had no idea how most boroughs were placed around London. Yes, you can easily find maps like the one on the right on Wikipedia, but it is hardly informative: it is definitely too hard to navigate it without any landmark. An image search for london borough map also gives almost always highly stylized maps that do not show much more than just the boroughs’ boundaries.

London borough map with tube stations

And honestly, there are not a lot of landmarks I would recognize outside of zone 2… wait, there are some: the tube stations! I rarely go outside zone 2 even with the tube, but I got used anyway to most names just by looking at the line maps while traveling. And this must be true for many people, because when I type “london borough map” on Google, on of the first completions it suggests is “with tube stations”. Sadly, I could not find any map with this information, so I created it! You can see it on the left as a png (click to enlarge and download), or get the pdf file. The main source is of course Tfl’s wonderful tube and rail map, and just a couple of hours of Inkscape did the trick.

The time it takes to change the time

Tue, 17 Feb 2015 00:00:00 +0000

You might remember from last year my map showing the difference between solar time and standard time; you might because it was by far the most shared content I created, and somebody even uploaded it to Wikipedia for the time zone article. But with great power comes great responsibility (…); in this case it means keeping the map up to date as the whims of the time regulators change. You would think this is a rare event, but when you consider the 200 or so sovereign states in the world, and the quadratic number of possible conflicts between them, it becomes a frequent event.

Indeed, the triggering event that prompted me to draw a new version wasn’t the bad color choice of the first attempt (for many people, green and red do not play well together), neither the small mistakes in the zone divisions, and not even some political faux pas (not marking Taiwan in the same way as most other sovereign states). After all, the map was drawn on top of an existing one found on Wikipedia, with just minor modifications apart from the gradients.

The real reason is that just a few months after releasing my map, Russia decided to change the time in most of the country. Since Russia extends to about 3.5% of the world’s area and 11.5% of the emerged lands area, the issue was too obvious to ignore.

Apart from this big change, the new map reflects that some territories in Ukraine and Georgia follow Moscow time instead of their countries’ timezone, and the introduction of the “Southeast” timezone (permanent UTC-05:00) in Quintana Roo, Mexico. Continental Mexico is about 30 degrees of longitude wide, from Tijuana to Cancun, so two timezones would make sense, instead it has four!

Solar time vs standard time

Overall, the map is still skewed towards the red (which means that the solar noon is later in the day), and most of what could be observed from the first version is still valid. In particular, for the joy of reddit commentators, China is still very red.

Sadly, one change did not happen: Australia still refuses to make Central Western Time, aka the awesome UTC+08:45 timezone, official. You see how the difference between Western and Central Australia is a whole hour and a half? This is not a big problem because no people live on the transition zone, apart from a very narrow strip of 350km along the southern coast, on the road between Perth and Adelaide, where about 200 proud people live on their own time.

If made official, it would be the timezone with the fewer residents (followed by UTC+10:30, Lord Howe Island, with about 360, and UTC+12:45, Chatham Islands, with about 600), apart from the ephemeral UTC-12:00 timezone: only a few birds inhabit the two US islands of Howland and Baker, that are the only emerged lands in it, with their whopping 4 squared kilometers. But, since they are uninhabited, nobody ever said what time they should follow, and they are unofficially UTC-12:00 just because they happen to be at the correct longitude..

I mentioned that 90 minutes is an unusual large difference between neighboring timezones, since of course the usual is one hour. But there are many places with much larger differences: these always involve China, thanks to its single, big, timezone. The worst offender here is the 90 kilometers border between Afghanistan (UTC+04:30) and China (UTC+08:00): a difference of three hours and a half!

How to draw a map

Drawing the first version of this map was a very long process done more or less by hand, modifying the source of an existing SVG file, and I definitely did not want to go over the same process again (and again when something else changes). So I blatantly ignored xkcd and proceeded to Create the Tool that would Create the Map for me. The real value of the tool for the future depends on the availability of up to date data, still I can trick myself into thinking I saved time.

All the tools I will mention in the following can be found in the solar time vs standard time repository.

Looking at a map, people often do not realize how many ingredients combine to create it; we take a lot of things for granted. This map has probably a sub-average number of ingredients, though we can see:

timezone boundaries over the sea (approximate, as the territorial water boundaries are too intricate to draw);
land/water and country boundaries;
timezone territories (which are a different hierarchy than countries: a country might have more than one timezone, and a timezone might encompass more countries);
cities coordinates;
labels of:
- countries;
- cities;
- timezones.

Luckily, there are datasets available for most of these; some with a unique source, maybe slightly out of date, some with more sources. It turns out that choosing the best in terms of accuracy and license is an important step when creating a map programmatically. In our case, timezones come from Eric Muller’s website, and the same for country boundaries (these in turns come from the FIPS dataset). Homogeneity is an important quality, and since timezone boundaries is a very rare dataset, it made sense to take the country boundaries from Eric as well. Coordinates of major cities (with other info useful to select whose to show) instead came from the ESRI datasets.

But getting the data is just the first step. The second is not very hard: draw the SVG with the boundaries. This involves some templating to create the file and most importantly translating latitude and longitude into pixel coordinates (that is, the choice of a projection). Again, I was constrained to a rectilinear projection because drawing the gradients was much simpler, so I sticked with the Miller.

Third step, writing the labels. Here there is an important choice: is it faster to design an algorithm that tries to place the labels automatically, or to place them semi-manually? The first option is not that easy: I would frame it as an optimization problem, where the function to optimize depends on the distance between the labels and what they refer to, on their size, and on how much they overlap, but I admit I did not try this. Instead, the semi-manual approach consists in placing the labels in a reasonable position (at the centroid of the country, and anchored to the coordinates of the city), and then to tweak the labels that need tweaking.

Fourth step is drawing the labels and lines of the timezones. For the labels, I just wrote a list of positions I wanted, so again very manual. For the lines, in theory a good approximation could be just drawing 24 equispaced meridians, but the map becomes very hard to read. To make easier the job of drawing a lot of polylines on a map, I wrote a very simple helper tool based on Maps API, that allows to draw directly on the map and retrieve the coordinates of the vertices in a JSON format.

Finally, the last step: data correction! Eric Muller’s data sadly is not completely up to date. Some fixes are easy (just changing the offset of a few timezones), others are not so simple, and I eventually decided to just use GIMP to draw over the final image generated by the program (which explains the GIMP file in the repository). Shifting the center of the map so that the cut point is not over Siberia was also done in postprocessing; in theory it should be easy to do at the projection stage, but the source datasets already split all lands on the 180 meridian, which made more convenient to use the same cut point in the projection.

A new year

Mon, 26 Jan 2015 00:00:00 +0000

A new year began, and I’ve been working on a new home and a new design for The poor man’s math blog.

My old home page.

I started this blog six years ago; I admit it is not a great achievement as the frequency has always been from low to zero, but I put a substantial amount of effort and time in it, and occasionally has been quite rewarding. The problem is that these moments are exactly those in which I created some issues to our beloved macchinisti, the administrators of the student-run server that hosted my blog until yesterday. Thanks a lot to them for bearing with me!

Because of this, I decided to give them a break and my blog a more stable home, on GitHub. To avoid making this post completely useless, I will describe my set up, in case it is of interest to anybody.

GitHub offers free hosting for static pages at addresses like <username>.github.io, based on the content of a repository with the same name. Not only this, but it also treats some specific files in that repository in a special way, think of if “compiling” the repository to a static website. For reference, this is the repository for this site.

I acquired poormansmath.net some time ago, but it was not possible to use it in my previous home because I didn’t own the domain; luckily, this is true for GitHub. The first special file we will see is indeed CNAME, that tells GitHub the domain you want to use for the site.

Speaking about the compilation to static pages, GitHub uses Jekyll, I think triggered by the presence of a _config.yml file. Jekyll allows a limited template engine to run on your file, and a Markdown interpreter on your posts. The advantage over a completely static site is that a lot of boilerplate can be factored out in templates, and you can write posts in Markdown instead than in raw HTML, which is nice. I forked a project I liked, from Barry Clark, and built on that.

I changed some typeface, added semantic information for crawlers, sharing buttons, and a very simple lightbox-like image viewer. This all within GitHub’s compilation with Jekyll. I need to specify this, because Jekyll in general is quite versatile, and you can add new functionalities in Ruby. But, of course, running untrusted code on your server is complicated, so GitHub restrict the compilation to a safe subset of Jekyll.

But I added also a couple of features outside of Jekyll. The first is commenting: the way to add it to a static site is to use an external service, and I chose Disqus. The second is a custom script to append to each post a list of related posts.

The actual export of posts and comments was a bit painful. You can export them from Wordpress, and try to convert them in Markdown, but a lot of things had to be revised manually. This is where Emacs rocks: powerful regular expressions, even across files, the awesome multiple cursors feature, and keyboard macros, all of them made the job much faster.

Le parole che caratterizzano un’epoca

Fri, 17 Oct 2014 00:00:00 +0000

“Le parole sono importanti!”, tuonava Michele Apicella alla giornalista di Palombella rossa, colpevole di usare espressioni come “kitsch”, “alle prime armi”, “cheap”. [Continua sul Post.]

If it looks like a word, it is a word one third of the time

Mon, 25 Aug 2014 00:00:00 +0000

With the last post, we looked at words of three letters in the Google corpora for six languages, to understand which language has the most three-letter words. But our original question was which one has the most ratio of words among trigrams that “look like” real words, and today we are going to answer that question.

Defining what looks like a real word might be difficult, because the answer is ultimately subjective, but we can again perform a simple statistical computation to get a good enough approximation.

I decided to define that a trigram abc looks like a word if:

the trigram abc occurs in any part of any word;
the bigram ab occurs as the starting part of any word;
the bigram bc occurs as the ending part of any word;

We could use again the Google corpora, limiting ourself to “common” words, but since the definition uses occurrences in any words, it should be enough to look in a dictionary. Also, to make everything a little more robust, I require that there are at least three occurrences of the trigram or bigram within the dictionary, not only one.

Using the script extract.py in this GitHub gist, we construct for each language a list of trigrams that look like words, starting from the standard dictionaries provided with Ubuntu. These are the number of such trigrams by language.

Language	Trigrams that look like words
American English	3364
British English	3366
German	2492
Spanish	2065
Italian	911
French	1911

English has the most of these trigrams, followed by German, Spanish, French, and Italian. The variance is very high; judging from my knowledge of the two extremes (English and Italian), I can argue that Italian is more strict in the order that letters can follow each other (usually alternating consonants and vowels). [¹]

To obtain the ratios, we just need to use merge.py in the same gist to intersect the list of trigrams with the list obtain via Google nGrams. Here are the results by language.

Language	Trigrams looking like words	Three-letter words	Ratio
American English	3364	1026	30%
British English	3366	1034	31%
German	2492	679	27%
Spanish	2065	529	26%
Italian	911	398	44%
French	1911	615	32%

Apparently, I was wrong! It’s not English that shows a high ratio of three-letter words but… Italian. This of course is mainly due to the much lower number of trigrams in Italian. It is also interesting to see that German is close to be the language with the lowest ratio despite being the one with the most frequent usage of three-letter words; but we already saw that that is because its most frequent three-letter words are really frequent.

Part of this series

I wonder if this has any consequence on how easy is to define a crossword in the two languages. Sadly I could not find any good source on this fundamental issue, I think also due to the lack of multilingual crossword authors. The Wikipedia entry shows examples of a Japanese-style grid, which seems very influenced by the stricter alternation of consonants and vowels. By experience, that is also the style of many Italian crosswords, but not necessarily the unique. ↩

If it looks like a words, then it is a word (?)

Mon, 07 Jul 2014 00:00:00 +0000

Being born as a native English speaker is a great luck these days. For the simplicity of its grammar and a predominance in the cultural world from the twentieth century onwards, English, or more precisely one of its dialect is the de facto international language, and being able to speak it natively is a huge advantage. If you are like me, instead, no matter how expert you may become, your level of comprehension of the subtleties will always be lower than that of any native speaker, you will always retain at least a bit of your original accent; you’ll never be able to pronounce all the words in the correct way, and there always will be some word you won’t know.

Among these, the biggest issue with English is its inconsistent pronunciation. Since even native speakers in the same country cannot reach an agreement on how to pronounce a word [¹], you may get away with mistakes by claiming that your weird pronounce is actually used by some native English dialect [²].

Something you cannot get away with, though, is the lexicon. English has a huge lexicon. Similar to ancient Romans, English conquered and assimilated, not only people and land but words [[^3]]. I jokingly stated that in English almost any sequence of three letters that look like a word is actually a word. Is this just a joke or is there some truth behind?

One could look into a dictionary to find out, patiently looking for all three-letter words. This is a fair strategy, but dictionaries are a static standardization of a living language, not dissimilar to what a picture is to a person. In particular, dictionary curators are aware that the actual number of English words is much higher than the few hundred of thousand listed in the most thorough dictionaries. Moreover, dictionaries only list one word for each “version”, for example for verbs.

Luckily, Google took a much finer snapshot of the living person that is the English language, by recording all the words in many books from the sixteen century onwards. Google collected millions of books, with billions of pages and about one trillion of unique English word occurrences, in the setting they were actually used. There are downsides here too: mainly the mistakes by the OCR software converting the pictures to the text; also, this is still not accounting for all the words in the language: think about all the colloquial terms, the jargon, that people often use speaking with their friends or colleagues, but that will never be written down, and definitely not in a book!

Building the frequency table

So, let’s play with the Google corpus. The first thing to notice is that it is very large. The list of words in American English starting with “a” takes about 1.5 GB uncompressed. But we are interested only in three letters words and in recent books (let’s say from 1980 onward) so we can download and filter these files at the same time with bash and a simple python script (sadly grep and python2 are easily confused with unicode characters):

for x in {a..z} other
do
  wget -O - http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-us-all-1gram-20120701-$x.gz \
    | gunzip -c \
    | python3 filter.py \
    >> three_letters_words_en_us.txt
done

where filter.py is shown in this GitHub gist.

The resulting file weights just 17 MB, much more manageable. Another short Python script (count.py in the same gist) takes care of grouping the words and computing the frequency (based on the total number of scanned words, that we extrapolated from the total_count file in the Google Ngram website).

The results

The resulting words for six languages are listed in the first sheet of this Google Sheet in decreasing order. All frequency values are shown as base ten logarithms, so a value of -1 means that that word (not differentiate between upper- or lowercase) was used once every ten words, -2 means every one hundred, and so on. The data is rearranged in the second sheet to show for each language how many words pass a certain frequency threshold, and in the third sheet there is a comparison with all possible trigrams.

The first thing to notice is that this corpus is not perfect: due to misattributions and mixed language books, you can find the among the top Italian words, and several other errors.

Also, due to mistakes in the recognition and to the size of the corpus, there are a lot of occasional “words” used only a handful of times. For this reason, I clipped the results to show only words that are more frequent than one in one million words (that is, that have a log probability greater than -6). Still, at the bottom there is a lot of garbage. The correct way to read the table and the graphs is then to decide a minimum frequency, above which we consider those “words of the language”. By manual inspection, the values in which the transition happens is between -4.5 and -5.0.

Number of words

Number of words above a certain frequency

To see which language has the most three-letter words, we just need to see which line is highest around the 4.5-5.0 mark on the graph in the fourth sheet (that you can also see here on the right). The two lines for English (British and American), which are obviously very similar, are always well ahead of neolatin languages (Italian, Spanish and French) [³]. German is ahead at the beginning, meaning that it has more very popular three-letter words, but loses its stamina as it progresses to rarer words, eventually aligning with neolatin languages.

Words usage

Cumulative usage of the top n words

Let us look now at words usage instead, that is, which proportion of words in actual books are three-letter words. The answer is in the graph in the fifth sheet and again copied here on the left.

Because of the strong start highlighted in the previous section, German leads: when you speak German, one in five word has three letters! English is around the 15% mark, and neolatin languages are just around 10-11%. It is interesting that the proportion of the top five German three-letter words (der, die, und, von, den) is more or less the same of the proportion of all Italian three-letter words.

The original question

The question we started from was a different one, though: if it looks like a word, is it a word? I collected some preliminary results on which percentage of trigrams are actual words in the considered languages in the third sheet (and in the last corresponding graph).

Percentage of three-letter words over all trigrams

The answer really depends on what we consider a letter: English is easy, 26 letters, but other languages are more complicated. Italian as spoken and written nowadays, for example, has 21 “base” letters, and 7 additional letters with diacritics: à, è, é, ì, ò, ó, ù: of these, only two have actual phonetic meaning (i.e., there is a phonetic difference between grave and acute accent), the others just mark the stressed syllable when it is the final one. Just a hundred year ago Italian had at least one base letter more (j) and some more diacritics variation. So, how many trigrams are there in Italian, 21 to the third, or 28 to the third? I think that the correct answer is the first, but not really sure, so I left both versions in the sheet.

Anyway, comparing the used words with all possible trigrams, it turns out that the language with the highest percentage of real words is… Italian! Not considering letter variants, Italian wins with 1.32%, over English with 0.9%. If we instead consider letter variants as different letters, then English wins with, no wonder, still 0.9% over Italian with 0.56%.

This is really a rough approximation of the original question, as definitely not all trigrams look like words! As this post is already too long, I will try to answer to our original question, by counting how many trigrams look like a word, in a future post.

Part of this series

I am not talking about accents here, these are different dialects. ↩
Admittedly, you will sound weird anyway, as you will invariably make mistakes in different directions and your sentence will be a mix of different dialects. ↩
In particular because also the y axis is logarithmic, so at -4.5 Italian has only three-quarter of American English three-letter words, and Spanish just 60%. ↩

Global time, emotional time, and the annoyance of DST

Mon, 02 Jun 2014 00:00:00 +0000

My post on the difference between local time and solar time obtained some mild popularity, and this gave me some time to think more about the subject of local vs. global time, and about emotional time. What is emotional time? Read on to know!

Solar time vs standard time

One of the main reaction to my map was surprise about the predominance of red areas (where the sun peaks after twelve o’clock) versus green areas (where the peak is before 12:00). This is easy to understand, if you think about it: would you prefer to live in a place where the sun shines from 05:00 to 17:00, or where it shines from 07:00 to 19:00? I doubt that many people would prefer the first (green) alternative to the second (red). [¹]

Areas observing DST

This explains why the majority of people is in a “red” part of the world, even without considering daylight saving time [²], that during the summer shifts most of Europe and Northern America one hour towards the red. [³]

Ok, so people prefer red places, and even more in the summer. But the way people name a certain instant is really a convention, so why, instead of shifting everybody’s clocks, don’t we just let everyone chose the hour they wake up, open their shops, go to school and to sleep? Changing the hours of operation only for the businesses for which it makes sense would probably cause less problems than shifting hours for everybody, after all.

The real problem with all of this is that people have a certain emotional “attachment” to a specific time in the day: for most people, 06:00 is quite early, and they’d feel grumpy if they had to wake up at that hour. Dinner time is also deeply felt by most people, deriving from family habits: for most Italians, it is from 20:00 to 21:00, increasingly progressively going south. In England, it’s around 19:00. It is probably easier to change the underlying time than to convince people to modify their dinner habits to follow the sun.

This is what I call emotional time: the feeling, learned with experience and culture, that a specific hour is the best for a certain activity. What is the perfect time for dinner? Of course is 20:00, dinner time at my parents’ house when I grew up. What if one morning I wake up at 11? Well, certainly I was lazy. Emotional time has almost no connection with solar time: we eat at 20:00 all year round, even if that means we eat in the dark in December and before sunset in June.

A global time?

As distances around the globe become shorter thanks to the internet, the need for fixing appointment from different timezones is increasing. If you correspond with just one or two other timezones, it is usually simple to remember one or two differences. [⁴]

Since emotional time varies only slightly around the world, it is then easy to understand which hours are reasonable for the other party. Just add the difference and compare with your emotional time, which hopefully is not very different from that in the other country.

Swatch Internet Time

Still, as the number of timezones to consider increases, this becomes more and more awkward, and there have been several attempts to create a global time: a time system in which an instant is called in the same way anywhere in the world. In 1998, Swatch introduced the Swatch Internet Time, where a day is divided in 1000 beats, each of them slightly shorted than 1.5 minutes. Swatch built upon the valid attempts of converting people to decimal times during the French Revolution, but added the “global time” part. [⁵] New Earth Time is a similar concepts of a global time, but based on angles instead of decimal systems, and hence every day is subdivided in 360 parts. Globetime is a more recent project that has been brought to my attention, and uses the same subdivision.

Of course, using UTC time would be a similar way of using a unique time around the globe, but changing the unit (to decimal or sessagesimal) makes it easier to avoid confusion when switching between local and global time. But does a global time helps in arranging meetings? Yes and no: if you are arranging in person it might help to refer to a common global time that everyone has learned to tie to their local time; if instead you are proposing a meeting, than you need to learn the usual working hours of the other party, which is, if any, more difficult than remembering just the timezone difference and relying on the canonicity of emotional time. [⁶]

Even more so, using a global time would mean that people had to lose their emotional connection with specific times, and the evidences show that this is hard for people, more than changing currency [⁷], and probably harder than changing measurement system.

Suboptimal LaTeX #5: miscellanea

Sun, 25 May 2014 00:00:00 +0000

This post concludes the short series on LaTeX; we will see some other suggestion that did not fit in the other posts on generic mistakes and suggestions, spacing, math environments and mathematics.

The source with example from this post is in a writeLaTeX.

Use what LaTeX gives you (part 2)

Even though the LaTeX-savyness of a person reading a post like this must be higher than normal, I’m still pretty sure that there is at least someone that does not know that there is no need to design a title page by yourself: all decent classes handle the command \maketitle, which takes the information from \title, \author and \date. Oh, and for the last one, you can use \today. Most classes also have an \abstract macro.

Another surprising hole in the LaTeX knowledge of some author is defining new macros with \newcommand: they are probably old TeX users, and therefore use \def. The main advantage of \newcommand is that it gives an error when the new macro’s name already exists, instead of silently overwriting it. Even if you do not use the existing macro, some other package might be relying on it, and weird errors may arise. If instead you are sure that there are no bad interactions, you can force the overwrite using \renewcommand.

Labels and refs

(ref) and eqref

It might comes as a surprise, but I’ve also witnessed (fortunately, just two) authors not knowing about automatic references with \label and \ref. Fixing their articles has been… interesting. Much more common is not knowing about \eqref for citing equations: writing the parentheses by yourself achieve a similar looking result, but using a semantic command can be quite useful in the typesetting phase. Speaking of \ref and \eqref, it is essential to use a non-breaking space before them, to avoid having the number as the first character of a new line. To do so, just change the space with a tilde like in this example:

... as said in figure~\ref{fig:x}.

Another frequent problem is to use the same label twice: keep an eye for LaTeX Warning: Label `x' multiply defined errors in the LaTeX logs. To mitigate the possibility of defining multiple labels, it is common to prepend the type to all labels names, for example sec:cats and fig:cats for a section and a figure about cats, instead of just cats… for both.

Many authors also do not know that LaTeX is particularly tolerant with respect to a wrong position of the label command! For example, a snippet like the following most probably does not do what the author wanted.

\begin{figure}\label{cat_picture}
\includegraphics[width=\textwidth]{cat_picture.jpg}
\caption{This is a picture of a cat.}
\end{figure}

Did you spot the problem? The numbering of the picture is given by the caption, not by the figure environment. Hence the label will not refer to the number of the figure, because it wasn’t defined yet! Instead, it will refer to the number of a previous element (like a section). You would assume that LaTeX gives you at least a warning for this suspect arrangement, but you’d be wrong. It is a good habit to write the label at the last possible time, even when not strictly necessary, in order to avoid these errors.

Theorem environments

If you need to use theorems or similar environments, the short story is to use the amsthm package. It allows you to define in one simple line the environment you need in your paper, for example:

\newtheorem{thm}{Theorem}

for a Theorem environment that you can use with \begin{thm}. Remember that there are three predefined styles of such environments, one well suited for statements to be proved (theorem, lemmas, etc.), one for definitions, examples, etc., and the last for additional notes and remarks. For example, this is a good preamble:

\theoremstyle{plain}
\newtheorem{thm}{Theorem}
\newtheorem{lem}{Lemma}

\theoremstyle{definition}
\newtheorem{defn}{Definition}
\newtheorem{exmp}{Example}

\theoremstyle{remark}
\newtheorem{rem}{Remark}
\newtheorem{note}{Note}

Theorems with amsthm

It also provides many other configurations, like whether to show the numbering and when to reset it.

Finally, it has an embedded proof environment, and a command \qedhere to help LaTeX drawing the q.e.d. symbol at the correct place: indeed, if you finish your proof with a displayed equation, LaTeX by default places the symbol on the next line.

Part of this series

Suboptimal LaTeX #4: mathematics

Mon, 10 Mar 2014 00:00:00 +0000

In the previous episodes: generic mistakes and suggestions, and how to fix the spacing and math environments in your papers. Continuing along these lines, in this post we will see some common mistakes done when writing mathematics.

As for the previous episodes, the source for all figures in this post is in a writeLaTeX.

If you learn one thing from all of this, let it be this

Every time I see a paper containing this mistake I cry a little bit: it is probably one of the most unprofessional errors that writers of scientific content with LaTeX can do. What is it? It is writing multi-letter symbols and even — please world forgive them — entire words in math mode without any escaping. It is simply horrible.

Text in math mode

What happens is that whenever you write letters as they are, LaTeX thinks that each letter is a symbol on its own. “This is bad” in the figure is rendered as if each letter was a symbol and you were multiplying all of them together (of course, spaces disappear). I have seen this error in many situations, such as (from the less ugly to the ugliest):

multi-letters symbols like the matrix spaces SL, SO;
functions whose name is an abbreviation of a word, like det or perm;
entire words;
or even sentences, like in \{x \mid x is odd\}.

In the case of multi-letters operators and functions (and, if you want, single-letter operators), you need to treat the letters as a single mathematical unit, with the spacing around them determined by their role. So for operators, you must use \DeclareMathOperator; functions are a bit blurred: I use \DeclareMathOperator too, but also \mathrm is acceptable.

In the other cases, you are inserting some text in mathematical mode; the correct way of doing that is using \text (available in the amsmath package), which instruct LaTeX to move temporarily back to text mode. Using \mathrm is wrong because it just changes the shape of the letter, not their semantic (for example, spaces are ignored). Also, \text draws the letters with the shape of the surrounding environment: if you are in a theorem, written in italic, then \text will write in italic too. And finally, it respect font sizes if you use it, say, within an exponent.

When to use `\mathrm`, and the myth of LaTeX taking care of everything

Apart from the situation presented above (multi-letter function names), there is at least another important case where you should use \mathrm: in the infamous “dx” at the end of an integral, the “d” should be upright… ok, maybe this is just my opinion, but it should!

Integrals and dx

Less debatable is the fact that LaTeX does not know that your “dx” has a different meaning in the integral than the function to integrate, and thus should be properly highlighted by leaving some space before it. I use the smallest standard space available, \,.

More in general, a lot of people support the myth that LaTeX knows better and typesets formulas in the best possible way. This is often true, with one big exception: when you, as a human, have more information about the formula than LaTeX. In that case, you should feel free to slightly tweak the formula. The most important thing to remember is to be consistent; in particular, it helps to use only the standard spacing macros (\!, \,, \;, \quad, \qquad).

The same applies to bracket sizing: leaving the decision to LaTeX, by using only \left( and \right), is certainly better than using only ( and ); but sometimes it is even better to decide for yourself. To do so, use \<size>l( and \<size>r), where size can be big, Big, bigg, and Bigg — and do not forget the l and r to specify if you are opening or closing the bracket! The same is possible for all other parenthesis types, including \langle and \rangle, that are almost always what you want instead of < and >.

Do you know about `\allowdisplaybreaks`?

Having a single equation going on for pages and pages is probably not the most compelling way of writing a paper… but sometimes it is necessary. By default, LaTeX does not break a page within a mathematical environment, so how do you avoid going over the page? Simple, you just break the environment in two!

Simple and wrong, because then you have to change your equation if the text before it gets longer or shorter (but that would be fine if you do it in the typesetting phase), but also because the alignment gets lost. If all the lines in the first part are very long and all those in the second part are short, the second part will be more or less center aligned, instead of left aligned. And so the reader loses the visual cue hinting that all lines were part of a single group.

The correct way is instead using \allowdisplaybreaks, which does what it says: allows LaTeX to break pages within a mathematical environment. You can use it just after the \begin{environment} to apply it only to that display. Moreover, if you end a line with \\* instead of \\, LaTeX won’t break after that line, which is very useful when you have a collection of equations with multiple lines.

Use what LaTeX gives you

A lot of people just use ... when \dots is visually and semantically better. Most don’t know about \cdots which is even better than \dots within a chain of additions, for example. I have very rarely seen \overbrace and \underbrace in the wild, and the same goes for \xrightarrow, to draw arrows with text on top. Nobody uses the cancel package to show that two terms were the opposite of each other. sfrac is almost unheard of, so instead of having beautiful diagonal fractions inside the text, we have either horrible 1/2 or, even worse, huge gaps between lines.

Even without resorting to obscure packages, that often causes problems when sending out your sources, LaTeX mathematics need not to be dull! You just need to know what symbols, accents, decorations, packages you can use.

The poor man's math blog

Some photos and statistics

Filling a void

The time it takes to change the time

How to draw a map

See also

A new year

See also

Le parole che caratterizzano un’epoca

If it looks like a word, it is a word one third of the time

Part of this series

If it looks like a words, then it is a word (?)

Building the frequency table

The results

Number of words

Words usage

The original question

Part of this series

Global time, emotional time, and the annoyance of DST

A global time?

See also

Suboptimal LaTeX #5: miscellanea

Use what LaTeX gives you (part 2)

Labels and refs

Theorem environments

Part of this series

See also

Suboptimal LaTeX #4: mathematics

If you learn one thing from all of this, let it be this

When to use `\mathrm`, and the myth of LaTeX taking care of everything

Do you know about `\allowdisplaybreaks`?

Use what LaTeX gives you

Part of this series

See also

The poor man's math blog

Some photos and statistics

Filling a void

The time it takes to change the time

How to draw a map

See also

A new year

See also

Le parole che caratterizzano un’epoca

If it looks like a word, it is a word one third of the time

Part of this series

If it looks like a words, then it is a word (?)

Building the frequency table

The results

Number of words

Words usage

The original question

Part of this series

Global time, emotional time, and the annoyance of DST

A global time?

See also

Suboptimal LaTeX #5: miscellanea

Use what LaTeX gives you (part 2)

Labels and refs

Theorem environments

Part of this series

See also

Suboptimal LaTeX #4: mathematics

If you learn one thing from all of this, let it be this

When to use \mathrm, and the myth of LaTeX taking care of everything

Do you know about \allowdisplaybreaks?

Use what LaTeX gives you

Part of this series

See also

When to use `\mathrm`, and the myth of LaTeX taking care of everything

Do you know about `\allowdisplaybreaks`?