Command window basics: variables, ans and display

Type -2^2 at Octave’s prompt and it answers ans = -4; type 2^3^2 and it answers 64, not the 512 of a maths book. The prompt evaluates every line by a few fixed rules, and the same rules decide what a script computes, so they are worth learning before anything else.

Below you follow a line through the prompt, predict how Octave groups operators and reads spaces inside brackets, create, list and clear variables, tell what is stored from what is shown, repair names that hide built-in functions, and use Octave’s own help to answer questions without leaving the command window.

The prompt evaluates each line, shows its value and keeps it in ans

Octave’s prompt in a terminal reads octave:1>, numbered: each line you type is evaluated, its value is kept, and the value is shown unless the line ends in a semicolon. A value without a name of its own goes into ansans: The variable in which MATLAB and Octave keep the most recent result that has no name of its own. An assignment such as x = 5 leaves it unchanged.. Follow two lines through it:

The session below types those lines and four more. Each line follows the prompt that read it, and what Octave printed comes under it:

from session import prompt, version

version()
prompt("prompt_basics.m")

# GNU Octave (aarch64-apple-darwin25.4.0) version 11.3.0
$ octave-cli --quiet
octave:1> 3 * 4 + 2
ans = 14
octave:2> ans / 2
ans = 7
octave:3> area_m2 = pi * 0.006^2;
octave:4> area_m2
area_m2 = 1.1310e-04
octave:5> area_mm2 = area_m2 * 1e6
area_mm2 = 113.10
octave:6> ans
ans = 7

Three rules show in it:

• A value without a name goes into ans. 3 * 4 + 2 gave ans = 14, and ans / 2 used it to give ans = 7.
• An assignment keeps the value in its name, and ans stays as it was. After area_m2 = … and area_mm2 = …, typing ans still showed 7.
• A semicolon at the end hides the value. area_m2 = pi * 0.006^2; showed nothing; typing area_m2 alone showed area_m2 = 1.1310e-04. A script follows the same rule, so a script without semicolons prints every value it makes.

Power binds tighter than unary minus, and powers group from the left

Before computing, Octave groups the operators of a line by operator precedenceoperator precedence: The rules that decide which operators of an expression are applied first. In MATLAB and Octave, ^ binds tighter than unary minus, so -2^2 is -4.. Two rules surprise people: ^ binds tighter than a minus sign in front of a number, and operators of the same level group from the left. Step through the groups Octave forms for -2^2 and 2^3^2:

The prompt confirms both trees, and the last lines show the transpose ', which turns a row into a column:

from session import prompt, version

version()
prompt("precedence.m")

# GNU Octave (aarch64-apple-darwin25.4.0) version 11.3.0
$ octave-cli --quiet
octave:1> -2^2
ans = -4
octave:2> (-2)^2
ans = 4
octave:3> 2^-1
ans = 0.5000
octave:4> 2^3^2
ans = 64
octave:5> 2^(3^2)
ans = 512
octave:6> 2 * 3^2
ans = 18
octave:7> 1:3+1
ans =

   1   2   3   4

octave:8> x = [1 2 3];
octave:9> x'
ans =

   1
   2
   3

The Octave manual lists the levels from the tightest down, and MathWorks documents the same order for MATLAB:

Two rows explain the rest of the run: 2^-1 is 0.5000, because the minus belongs to the exponent, and 1:3+1 adds before it builds the range, so it gives 1 to 4.

Inside brackets, a space can start a new element

Inside square brackets, a space or a comma separates the elements of a row. That makes the spaces around a minus sign matter:

from session import prompt, version

version()
prompt("brackets.m")

# GNU Octave (aarch64-apple-darwin25.4.0) version 11.3.0
$ octave-cli --quiet
octave:1> a = 3; b = 2;
octave:2> [a -b]
ans =

   3  -2

octave:3> [a - b]
ans = 1
octave:4> [a- b]
ans = 1
octave:5> [a, -b]
ans =

   3  -2

octave:6> [a (b)]
ans =

   3   2

octave:7> (a -b)
ans = 1
octave:8> [1 +2]
ans =

   1   2

With a = 3 and b = 2, [a -b] is the row 3 -2: a space before the minus and none after it make -b a new element. [a - b] and [a- b] are the single number 1, because a minus with a space after it is a subtraction. [a (b)] is the row 3 2, two elements, not a indexed by b. The rule belongs to brackets: (a -b) in parentheses is the subtraction 1 again, and a plus sign follows the same rule as a minus, so [1 +2] is the row 1 2.

Names hold values, and the workspace lists them

Every name you assign at the prompt lives in the workspaceworkspace: The set of variables that exist at the prompt and in the scripts it runs. who and whos list it, and clear removes names from it. until you clear it or close Octave. A name starts with a letter, may hold letters, digits and underscores, and is case-sensitive. These lines compute the cross-section of a 12 mm rod and then look at the workspace:

from session import prompt, version

version()
prompt("names.m", errors=1)

# GNU Octave (aarch64-apple-darwin25.4.0) version 11.3.0
$ octave-cli --quiet
octave:1> force_kN = 15;
octave:2> diameter_mm = 12;
octave:3> area_mm2 = pi * diameter_mm^2 / 4
area_mm2 = 113.10
octave:4> who
Variables visible from the current scope:

area_mm2     diameter_mm  force_kN

octave:5> whos
Variables visible from the current scope:

variables in scope: top scope

  Attr   Name             Size                     Bytes  Class
  ====   ====             ====                     =====  ===== 
         area_mm2         1x1                          8  double
         diameter_mm      1x1                          8  double
         force_kN         1x1                          8  double

Total is 3 elements using 24 bytes

octave:6> exist('area_mm2')
ans = 1
octave:7> Area_mm2
error: 'Area_mm2' undefined near line 1, column 1
octave:8> clear area_mm2
octave:9> exist('area_mm2')
ans = 0
octave:10> isvarname('2nd_try')
ans = 0
octave:11> namelengthmax
ans = 63

What the session shows:

• who lists the names; whos adds each one’s size, bytes and class. All three are 1x1 double values of 8 bytes, 24 bytes in all.
• exist returns a code. 1 means a variable; after clear area_mm2 it returned 0, nothing of that name. The codes are the same in MATLAB: 2 means a file, and 5, a built-in function, comes back later in this lesson.
• Names are case-sensitive. Area_mm2 is a different name, never assigned, so Octave answered 'Area_mm2' undefined.
• Some names are not allowed. isvarname('2nd_try') is 0 because a name cannot start with a digit, and namelengthmax says names may have up to 63 characters.

clear with a name removes that name; clear alone empties the whole workspace, which is why a script that starts with clear cannot rely on anything typed before it.

Display is not data: format changes what you see, not what is stored

Octave shows numbers in a short format by default, five significant digits. The value behind the display keeps all its digits, and the next session shows how far apart the two can be:

from session import prompt, version

version()
prompt("display_demo.m")

# GNU Octave (aarch64-apple-darwin25.4.0) version 11.3.0
$ octave-cli --quiet
octave:1> p = pi
p = 3.1416
octave:2> format long
octave:3> p
p = 3.141592653589793
octave:4> format short
octave:5> p
p = 3.1416
octave:6> big = 123456789
big = 1.2346e+08
octave:7> big - 123456780
ans = 9
octave:8> v = [1 1000.5]
v =

   1.0000e+00   1.0005e+03

octave:9> none = zeros(0, 3)
none = [](0x3)
octave:10> ok = [3 7 9] > 5
ok =

  0  1  1

octave:11> class(ok)
ans = logical

• format long shows more digits of the same value. p showed 3.1416, then 3.141592653589793, then 3.1416 again; the stored value never changed. The Octave manual puts it plainly: the command “only affects the display of numbers, not how they are stored or computed”.
• A short display can hide the digits that matter. big = 123456789 showed 1.2346e+08, yet big - 123456780 is exactly 9: every digit was stored.
• A row with very different sizes is shown in e-notation. [1 1000.5] appeared as 1.0000e+00 1.0005e+03.
• Empty arrays show their size. zeros(0, 3) showed [](0x3): zero rows and three columns.
• Comparisons give logical values, shown as 0 and 1 with class logical.

MATLAB lays the same values out differently: its default adds blank lines around results, and MathWorks’ examples show a common scale factor above a row of very different sizes. So code that checks a result compares values, as big - 123456780 == 9 does, never the text that one program displays.

A name can hide a function: i, j, pi and sum

i, j, pi and sum are functions, and a variable of the same name hides each of them. This is called shadowingshadowing: When a variable or a file with the same name as a function hides it, so that the name no longer calls the function, as sum = 0 hides sum()., and it is the classic bug of the command window:

from session import prompt, version

version()
prompt("shadowing.m", errors=1)

# GNU Octave (aarch64-apple-darwin25.4.0) version 11.3.0
$ octave-cli --quiet
octave:1> z = 3 + 2*i
z =  3 + 2i
octave:2> i = 5;
octave:3> z = 3 + 2*i
z = 13
octave:4> z = 3 + 2i
z =  3 + 2i
octave:5> sum = 0;
octave:6> sum([4 5 6])
error: sum(6): out of bound 1 (dimensions are 1x1) (note: variable 'sum' shadows function)
octave:7> which sum
'sum' is a variable
octave:8> clear sum
octave:9> sum([4 5 6])
ans = 15
octave:10> exist('pi')
ans = 5

• i is the imaginary unit until you assign it. 3 + 2*i was the complex number 3 + 2i; after i = 5 the same line gave 13. The literal 2i stays complex whatever i holds, so write 1i for the unit and use names such as k, m or n for loop counters.
• sum = 0 breaks sum(...). With a variable called sum, sum([4 5 6]) asks for elements 4, 5 and 6 of a 1x1 variable. Octave 11.3 says so, and adds (note: variable 'sum' shadows function).
• which and exist find the culprit. which sum answered 'sum' is a variable, and exist('pi') returned 5, a built-in functionbuilt-in function: A function that is part of the program itself, compiled into it rather than read from an .m file, such as sum or det. exist returns 5 for one., because nothing hides pi in that session.
• clear name repairs it. After clear sum, sum([4 5 6]) gave 15 again.

help, lookfor and which answer questions without leaving Octave

Octave’s help is inside Octave. Which command to use depends on what you already know:

Here is each of them at the prompt:

from session import prompt, version

version()
prompt("help_tour.m")

# GNU Octave (aarch64-apple-darwin25.4.0) version 11.3.0
$ octave-cli --quiet
octave:1> which det
'det' is a built-in function from the file libinterp/corefcn/det.cc
octave:2> get_first_help_sentence('det')
ans = Compute the determinant of A.
octave:3> lookfor determinant
det                 Compute the determinant of A.
octave:4> help namelengthmax
'namelengthmax' is a function from the file /opt/homebrew/Cellar/octave/11.3.0_1/share/octave/11.3.0/m/miscellaneous/namelengthmax.m

 -- N = namelengthmax ()
     Return the MATLAB compatible maximum variable name length.

     Octave is capable of storing strings up to 2^{31} - 1 in length.  However
     for MATLAB compatibility all variable, function, and structure field names
     should be shorter than the length returned by ‘namelengthmax’.  In
     particular, variables stored to a MATLAB file format (‘*.mat’) will have
     their names truncated to this length.

Additional help for built-in functions and operators is
available in the online version of the manual.  Use the command
'doc <topic>' to search the manual index.

Help and information about Octave is also available on the WWW
at https://www.octave.org and https://octave.discourse.group/c/help/

which det says that det is a built-in function, written in C++ inside Octave, while help namelengthmax starts with the file it comes from, an .m file of Octave’s own. get_first_help_sentence('det') returns the one sentence lookfor searches, Compute the determinant of A., and lookfor determinant found det by that sentence. A help text starts with the call forms, here -- N = namelengthmax (), then says what the function returns. When lookfor finds nothing, lookfor -all searches the whole help texts instead of their first sentences. help alone explains how to use help, an operator needs quotes, as in help "+", and doc det shows det’s entry of the Octave manual, through the Info program when Octave runs in a terminal.

Practise: predict, then type

Octave itself shows whether you got each of these right, so do them once it runs on your computer:

1. Write down what -3^2, 2^-2, [5 -1] and [5 - 1] give, then type them at the prompt and compare.
2. Assign area_mm2 = pi * 12^2 / 4, then use who, whos and exist('area_mm2') before and after clear area_mm2.
3. Write pressure_table.m, which converts 1.013, 2.5 and 10 bar to kilopascals (1 bar is 100 kPa) and prints each with two decimals, 101.30 kPa, 250.00 kPa and 1000.00 kPa, with fprintf. Before you assign a name, check that exist returns 0 for it.

The session below asks up to twelve of this page’s questions, many of them “what does this print?”, in the order your answers call for, and explains each answer on the spot:

Interview questions

Warm-up: what does a semicolon at the end of a line do, and what is ans?

A semicolon at the end of a line stops Octave and MATLAB from showing the line’s value; the value is still computed and kept. Without it, the command window shows the result with its name, as in area_mm2 = 113.10. ans is the variable that holds the most recent value that had no name of its own: typing 3 * 4 + 2 sets ans to 14, and an assignment such as x = 5 leaves ans unchanged. In scripts the same rules apply, so a line without a semicolon prints its value in the middle of the output, and code that reads ans several lines later may get a value from another line. Give results you need again a name of their own, and end lines with semicolons unless you want to see the value.

Key takeaways

• The prompt evaluates each line, keeps a value without a name in ans, leaves ans alone on assignments, and shows every value unless the line ends in a semicolon.
• ^ binds tighter than unary minus and powers group from the left, so -2^2 is -4 and 2^3^2 is 64; write parentheses, such as 2^(3^2) for 512, whenever the order matters.
• Inside brackets a space before a minus starts a new element: [a -b] is a row of two, [a - b] a single difference, and commas say which you mean.
• who, whos and exist show what the workspace holds, names are case-sensitive and up to 63 characters long, and clear removes names.
• format changes only the display: 1.2346e+08 was all of 123456789 underneath, so code compares values, never displayed text.
• A variable named i, pi or sum hides the function of that name until clear removes it, and help, lookfor, doc and which answer questions from inside Octave.