Hierogyphic security

 

It’s been a while. Here’s another in my long-running, sporadic series of not very amusing drawings. Should be producing them more regularly now that I have a job with more meetings.

 

Glyphs.

Advertisements

$$ math $$ versus \[ math \] in LaTeX

Let’s say you want to display some maths in LaTeX. You don’t want to use equation and related environments.

Inline maths is enclosed between dollar signs, and is formatted as part of the sentence.

A displayed equation is enclosed between pairs of dollars or between \[ and \]. But $$ … $$ and \[ … \] are not the same. For one thing, \[ … \] responds to the documentclass option fleqn to align the equations to the left rather than center them. Thus, we get this:

Where ‘Inline’ is indented because it starts a paragraph. First two displayed equations use $$ … $$ and are centred, the last uses \[ … \] and is indented by a length called mathindent.

Here is the LaTeX code

\documentclass[fleqn]{article}
\begin{document}
Inline maths: a simple equation might be $x+y=1$.
The solutions to the quadratic equation
$$ax^2 + bx + c = 0$$
are given by
$$x=\frac{-b \pm \sqrt{b^2-4ac}}{2a},$$
which is to say
\[x=\frac{-b \pm \sqrt{b^2-4ac}}{2a}.\]
\end{document}

Notice the argument ‘fleqn’ has been given to the article class, but $$ … $$ did not obey it. \[ … \] did, but not all the way to the left. If you want the left aligned maths to be hard left, you need to redefine mathindent to be zero length:

\setlength{\mathindent}{0pt}

and this gives

Anyway.

 

Comparing PDFs; diff-pdf, pdftotext, diff…

Command line tool diff-pdf at https://vslavik.github.io/diff-pdf/ is a very handy tool. It basically superimposes two PDFs to make the differences show up. It’s not a textual comparison as such. Here is the command line:

$ diff-pdf --view test.pdf test_mod.pdf

The black bits are common to both files, the red is in one version and the cyan in another. If the differences result in new lines being inserted, the whole page turns blue/red, since the lines don’t match up any longer:

So it is a very good way of isolating minor changes (say, between consecutive proofs of the same document) and checking if two files are actually identical (though conventional diff can indicate whether two binary files differ or not). It’s less good for comparing and decided which version is ‘better’ since the result can look a bit messy.

Using conventional diff:

$ diff test.pdf test_mod.pdf
Binary files test.pdf and test_mod.pdf differ

Use of pdftotext on both PDFs, then using conventional diff is pretty useful, too. Here is the output from just such a test:

$ pdftotext.exe test.pdf
$ pdftotext.exe test_mod.pdf
$ diff test.txt test_mod.txt

1,3c1,5
< This is a very basic look at using METAFONT and gnuplot to make figures
< for use in LATEX. I am using Linux, but the same process ought to work for
< other LATEX environments; indeed, that ought to be one of its strengths. --- > This is an extremely basic look at using METAFONT and gnuplot to make
> figures for use in LATEX. I am using Cygwin, but the same process ought to work
> for other LATEX environments; indeed, that ought to be one of its strengths.
> Here is some text added to make the line wrap and offset compared to the other
> document.
36c38
< see. --- > see. Also, this document is really just to show use of diff-pdf.
42d43
< 5. If you like at this point you can try:
45c46,47
< $ gftodvi test2.600gf --- >
  5. If you like at this point you can try:
> $ gftodvi test2.600gf
78a81,84
1
> 2
>
>
  Figure 1: Here is my pointless plot.
83,85d88
< 2
<
<
  Figure 1: Here is my pointless plot.

No conclusion. Just noting that these tools are handy.

Ende.

Plotly on cygwin; the absolute basics

This has to be about the absolute basics. I don’t know anything else.

Plotly (https://plot.ly/) is an interactive, online graphing tool. It can be called from JavaScript, Python, whatever. This post is about getting it to work through Python on Cygwin.

This all mostly follows instructions on the Plotly website.

(1) Installed via pip. What’s pip? A Python package manager. I ran the Cygwin setup.exe program and made sure that Puython was installed (in my case it was 2.7) and then installed the matching pip2.7 (Cygwin package python2-pip). So installed that and all its dependencies.

(2) Opened a Cygwin terminal (not an X terminal, just mintty) and typed:

$ pip install plotly

and watched some magic occur.

(3) Went to the Plotly website and created my (free) account. Went to my account settings and selected ‘API keys’. Could not see key — just looked like a row of dots! But hitting ‘Regenerate key’ gave me a new, visible one. Copied that text and noted my username.

(4) In Cygwin, (note, $ is the Cygwin prompt, >>> is the python prompt) typed:

$ python
Python 2.7.13 (default, Mar 13 2017, 20:56:15)
[GCC 5.4.0] on Cygwin

>>> import plotly
>>> plotly.tools.set_credentials_file(username='DarrenG2', api_key='<<insert your key here>>')

quit()

This set up the info needed for the local Plotly/Python installation to talk to the website where the graph will appear.

(5) Checked that this had worked out. Back at Cygwin prompt, in home directory, typed:

$ cat .plotly/.credentials
{
    "username": "DarrenG2",
    "stream_ids": [],
    "api_key": "<<your key here>>",
    "proxy_username": "",
    "proxy_password": ""
}

(6) OK, looked good. Now, tested it by grabbing an example from the Plotly website. Created a file ‘plotly_example.py’ and pasted in some text copied from the website:

import plotly.plotly as py	
from plotly.graph_objs import *

trace0 = Scatter(
    x=[1, 2, 3, 4],
    y=[10, 15, 13, 17]
)
trace1 = Scatter(
    x=[1, 2, 3, 4],
    y=[16, 5, 11, 9]
)
data = Data([trace0, trace1])

py.plot(data, filename = 'basic-line')

(7) Then saved and ran the script

$ python2.7.exe plotly_example.py
High five! You successfuly sent some data to your account on plotly. View your plot in your browser at https://plot.ly/~DarrenG2/0 or inside your plot.ly account where it is named 'basic-line'

Looked good, though they’ve spelled ‘successfully’ unsuccessfully.

(8) But where was the graph? Well, I was working in a basic terminal window. It sent the graph to the web, but then tried to open it using the default links, the text browser. So all I got was a blank screen (typed ‘q’ to quit links). There are a couple of options to see the graph — one is just to paste the given URL into Edge, Chrome, Firefox. Another is to tell Cygwin to look elsewhere for its browser…

(9) Edited my .bash_profile file in my Cygwin home directory and added these two lines:

$ BROWSER=/cygdrive/c/Users/darren/AppData/Local/Mozilla\ Firefox/firefox.exe
$ export BROWSER

This set up the environment variable BROWSER and pointed it at the firefox.exe file (non-admin install, so in an unusual place). I also ran these two lines in the terminal window to save me closing and reopening it.

(10) Repeated step (7) and — lo and behold! — a Firefox window popped up and the graph was in it!

Plotly graph in Firefox, after running the script in Cygwin.

Plotly graph in Firefox, after running the script in Cygwin.

Now, mastery of Plotly and Python is a much bigger project, but at least this offers the beginnings. Note also that the graph can be edited interactively within the webpage where it appears.

 

Plots away!

 

I don’t exactly love Microsoft Word

Man I hate the revision pain pane. When Word does this weird fail (as shown below) it’s not always while the pane is open, but far too often it is. And this document is a paltry 7000 words — hardly a vast epic. The program really should be able to cope.

Word is not responding. Yes, that's the least of it.

Word is not responding. Yes, that’s the least of it.

So: How do I stop the revision pane from ever opening? I know if you show markup (simple or all) the pane pops up less often — but it still pops up sometimes. I’d like to permanently disable it, root it out from the program and never ever see it again under any circumstances.

Any thoughts?

 

Word word word.

Ghostscript and GSview without administrator privileges

First, the downloads. GhostScript from: https://ghostscript.com/download/

I downloaded gs921w64.exe, and double clicked, but Win 10 wanted admin credentials to run the installer. I changed extension to zip and double clicked. Extracted contents to

C:\Users\usename\install\gs

Then the viewer, GSview. To: http://pages.cs.wisc.edu/~ghost/gsview/index.htm

Again, install wanted admin rights, even when I did not specify C:\Program Files.

Again, renamed to zip, and unzipped into

C:\Users\usename\install\gsviewtemp

Then went in there and double-clicked setup.exe

Created and then set install directory to be

C:\Users\usename\install\gsview

Unclicked ‘for all users’ since no admin rights. Ran installer; it did stuff, but said something failed. Still, tried to run gsview64.exe itself (found it in the install folder and clicked on the exe file); it started but said it cannot find ghostscript; ok need to set some variables/config stuff.

So, GSview was running, but crippled. Went into its menus.

Selected ‘Option’ → ‘Advanced configure’ and put in the correct locations for ghostscript lib and dll. For ‘Ghostscript DLL’:

C:\Users\username\installs\gs\bin\gsdll64.dll

For ‘Ghostscript include Path’:

C:\Users\username\installs\gs\lib;C:\Users\username\installs\gsview\Ghostgum\fonts

(The fonts folder was not there; I just created it where GSview expected it. It’s empty.) Closed then reopened gsview64.exe; it opened without error messages. Opened a test file… (examples inside gs folder). Yep. OK, try printing to some writer… only found 3 devices… pdfwrite, mswinpr2 and djet500… where are others? Won’t worry for now.

Oh well, seems usable.

Added the ghostscript binary (bin) directory to my local user path. (See for example here for local path.) Might be useful to add the gs lib to the path as well. The lib folder is where a lot of the batch files (command line tools) live. Didn’t find any pfb files after the install (maybe that was the error the installer threw?)  so added the path to the type1 fonts in my MikTeX installation. Can’t hurt, probably won’t help.

Added the path to the GSview binary to my path as well, just so I can use it from the command line more easily.

Anyway, seems to work nicely.

Standard example file from Ghostscript, viewed using GSview installed without administrator rights on Windows 10.

Standard example file from Ghostscript, viewed using GSview installed without administrator rights on Windows 10.

Natch.

Wishful Drinking by Carrie Fisher: Spare me the details.

Yeah, it’s really funny. Reads very much like spoken word written down. Words fly by quickly, often ironic or mordant. It’s short, generously leaded, so probably not that many words. It’s like therapy bound into a codex and sold.

The cover of <i>Wishful Drinking</i> by Carrie Fisher.

The cover of Wishful Drinking by Carrie Fisher.

Fortunately, there’s not too much about Starwars, since I am over 12 years old and don’t care about it. It’s kind of sad how large it loomed in her life. It’s often struck me that being an entertainer is a funny sort of thing, from the point of view of fulfilment. Is helping people pass their time away satisfying? I guess the key thing, if you’re the reflective type, would be whether you feel that you’re enriching the viewers’ lives or just helping pass the time until the grave. But what value a laugh or a thrill? People love those movies, probably too much. What’s wrong with giving people something that they just plain really like? Nothing.

The book made me think about people with the same mental issues as Fisher but without the cushion of money or the spotlight of fame. I don’t know what’s worse, but it seems to me she could always afford and find a therapist, so maybe the money and fame might be preferable as a position to inhabit while battling demons. Also, you can write a book about it and people will read it ‘cos they’ve heard of you.

Her story certainly makes a strong case that it would be preferable to win fame after a few years in the real world, rather than spending your whole live in an unmoored bubble.

Funny. Honest. Worth the little time it takes to read it. Probably better on stage, but sadly it’s too late for that now. The self-destructive stories in the book take on a darker tone now that they’ve taken their tithe. Perhaps it’s not as funny as it would have been a little while ago…

 

Sad stories.

Random useful Word stuff

Notes to self.

(http://www.fileformat.info/info/unicode/category/Sm/list.htm)

(‘␣’ means empty space, eg one hit of the space bar; ‘Alt-x’ means hold down ‘Alt’ and press ‘x’. If a space is not marked with the empty space character, it is just used to separate commands and should not be typed.)

in Word, type ␣2212 Alt-x to get a proper minus sign, not an en rule or a hyphen; it sits at the right height and gives the right spacing. The space in front is not always needed, but may be needed to separate the ‘2212’ from the previous characters so Word knows what to apply the ‘Alt-x’ function to. Here are the other ones I find most useful:

Useful Alt-x codes for maths in Word

If in the ‘Advanced find’ dialogue I select ‘Use wildcards’ then the search differentiates between non-breaking spaces and spaces.

Just some stuff.

Other stuff.

 

 

 

 

Search insde Word, PDF, XML and other files—installing and using crgrep

I am an editor in a business that uses Micro$oft products, but I want to be able to use the Linux CLI tools with which I am moderately familiar. In particular, I want to be able to grep Word documents, and that’s a problem because the new Word file format chops the text up and zips it up and hides it away. I googled and read a bit about crgrep (‘common resource grep’). Here is my experience so far.

Downloaded from

https://bitbucket.org/cryanfuse/crgrep

or

https://sourceforge.net/projects/crgrep/

Created a subdirectory c:\Users\username\installs\crgrep and downloaded the zip file into it. Worked in Cygwin, hence the forward slashes and dollar signs in the following. This could also be done through the GUI or in a PowerShell or CMD window. Choice is a wonderful thing.

$ unzip crgrep-1.0.5.zip
$ cd crgrep-1.0.5/
$ vim INSTALL.txt

OK, so it needs java. Does it need the compiler (probably not, but check…). In the crgrep folder, typed:

$ grep -ir javac

Returned no results calling the javac compiler. So it looks like the program needs the runtime but not the development kit (JDK), so that’s good. It’s what you’d expect. Now, I have the wonderful ImageJ installed (works effortlessly in userspace), and it installs the Java runtime environment, JRE. Maybe I can use that.

Now, according to the INSTALL.txt file, the JAVA_HOME variable that crgrep wants points at something like

JAVA_HOME=C:\Program Files\Java\jdk1.8.0_xx

and my grepping told me that java.exe should be in %JAVA_HOME%\bin\java.exe

In Cygwin, my ImageJ tree looks like:

/cygdrive/c/Users/username/installs/ij/ImageJ/jre/bin

Which meant I needed to set JAVA_HOME to be C:\Users\username\installs\ij\ImageJ\jre (Windows-style path) (that is, the variable points the directory with the bin directory inside it, not the bin directory or the binary file itself.)

But first checked the version — needs 1.8.

$ cd ../../ij/ImageJ/jre/bin/

$ ./java.exe -version
java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)

OK.

I installed in userspace (work computer, no root/admin access), so I went to my Windows account settings (given the various versions of Windows, I’ll assume a user can find their own account setting page) and (for Win 10; Win 7 is no doubt different) in the ‘Find a Setting’ box I typed ‘env’ for ‘environment variables’, and chose ‘edit the variables for your account’. Note that searching for ‘path’ turns up nothing. It’s a little trick!

Added an entry to the path:

(Path → Edit → New)

c:\Users\username\installs\crgrep\crgrep-1.0.5\bin

and created a new environment variable:

JAVA_HOME=C:\Users\username\installs\ij\ImageJ\jre

And exited everything, esp. the command line window, then opened it again, typed SET in a CMD (‘DOS’) window to see if the new variables were present, then tried the command:

H:>crgrep --help
usage: crgrep [options]  []
crgrep: Common Resource Grep.
 -a,--text               Process binary files or database columns as if
                         they were text
    --color        Alias for 'colour'.
    --colour       Colour-highlight matched text ('always', 'auto'
                         or 'never'). Default colour is red, see USAGE.txt
                         for other colour settings.
 -d,--database           Database grep (disables file search)
 -h,--help               Help
 -i,--ignore-case        Ignore case distinctions in matched text
 -l,--list               List resources which produce a match by name. No
                         content is searched.
 -m,--maven              Include Maven POM file dependencies in search
    --mood    Only include matching content expressing a
                         specific sentiment; values include 'positive',
                         'negative' or 'neutral'. Ignored if -l specified.
                         Requires model data; see INSTALL.txt
    --ocr                Enable OCR text extraction from images; requires
                         tesseract libraries. See INSTALL.txt
 -p,--password      Password required to access a resource,
                         optionally used with -u
 -P,--proxy         Proxy settings for http access, specified as
                         [:]
 -r,--recurse            Recursive search into resources
 -u,--user          User ID or username required to access a resource
 -U,--uri           URI to specify a JDBC database resource
 -V,--version            Print the version number of CRGREP to the
                         standard output stream
    --warn               Display all warnings to standard output
 -X,--extensions    Enable one or more extensions; comma sep. list
                         such as -Xdebug,trace
If  is not specified, or is '-', read from stdin
Please report issues at https://bitbucket.org/cryanfuse/crgrep/issues

OK, promising.

I want it for grepping Word files, so let’s see… yes, it finds ‘data’ in the test file, and outputs a nice clean stream:

H:>crgrep data text.docx
text.docx:T:A key part of his research was the analysis of large
datasets. As part of this he developed a software suite that included
data modelling, reduction and correction techniques, and made of use the
National Computing Infrastructure and other supercomputers. He enjoys
the challenge of analysing and explaining complex data using words and
carefully designed graphics. He likes Linux and the LATEXtypesetting
system.

How about PDF? Converted the Word doc to PDF using the ‘Save as’ dialogue in Word. Then…

H:\>crgrep data text.pdf
text.pdf:1:36:datasets. As part of this he developed a software suite
text.pdf:1:37:that included data modelling, reduction and correction
text.pdf:1:40:challenge of analysing and explaining complex data using

Different output because of how PDF and Word chop up the text, but instances found in both cases. No need to specify a file type or anything. I have not explored the command line options, but I am already finding the program useful — for example, when I want to find multiple instances of multiple expressions (say acronyms or references) in multifile projects.

Kudos.

Just grepping around.