Skip to content

Latest commit

 

History

History
385 lines (261 loc) · 19.3 KB

File metadata and controls

385 lines (261 loc) · 19.3 KB

Meta Syntax


Pre-Processor Directives

Pyndent code syntax is 100% pure Python, as it must be. The meta-syntax, used by Pyndent pre-processor to rewrite the (Python) code indentation, uses a couple of "code-block delimiters" (by default C-like ones: "{" and "}", but they can be customized) and a few meta-statement (AKA pre-processor directives):

#delim <start_delimiter> <stop_delimiter>

where <start_delimiter> and <stop_delimiter> can be single characters (usually in pairs: "{" "}", "<" ">", "[" "]", but they can be litterally everything, as far as you follow Pyndent meta-syntax: "/" "\", "->" "<-", etc.) or even single words (e.g. "begin" "end", "block_start" "block_end", "apples" "bananas"). Multiple words are currently not accepted.

The only unusable character is "#" as it's both reserved for pre-processor directives, and will be used to comment out the delimiters in <source>.py output files (which also means "##" can't be a valid delimiter too).
You need to use #delim directive as soon as you like/need to use non-standard custom delimiters, as Pyndent needs to know how you (custom) delimited your code blocks, to be able to re-indent them.
Like all Pyndent pre-processor directives, #delim must appear at meta-source top (but NOT as the very first line, to avoid confusion with hashbang #!), for it to be used.

NOTE: in Linux environments, an #! (also called hashbang) is used to tell the shell which "processor" to call, to manage the "text" (source code) the shell is reading: that's why a #delim directive must avoid to be written in the first row: writing it starting in the second or further rows is considered correct: only important thing is a directive appears before the first place it need to be used. If you need to customize your delimiters, you can't add a #delim after the very first line where (default) delimiters are written, as that behavior will be flagged as inappropriate and made clear with a warning:

The following syntax works: #delim {{ }} is written before the line where {{ or }} appears for the first time:

#delim {{ }}
import sys

def example():
{{
print("example!")
}}

The following doesn't work: #delim {{ }} is written after the the first {{: pyndent will read {{, will recognize it's different from its default { and will just write {{ to <stdout> (or file, if you'll use -o switch), then Python will complain with you 'cause the brackets it's not managing correctly will more probably create syntax errors somewhere else, in (correct) Python code

import sys

def example():
{{
#delim {{ }}
print("example!")
}}

The rule is: when Pyndent will start using a delimiter (its own defaults or your custom ones) it won't accept to change it later (it will emit a warning like "WARNING [mixed-delimiters]: #delim directive used after first delimiter is used"): midex-delimiters are not allowed.


Directives Examples

#delim example 1:

#delim begin end
def example():
begin
    if True:
    begin
    print("nested")
    end
end

#delim example 2:

#delim < >
def example():
<
    if True:
    <
    print("nested")
    >
>

Hashbang management

In Pyndent meta-sources you may need, like in every other source, to use the hashbang (#!) to engage Pyndent, instead of explicitly calling it externally. So, you're going to have a first meta-source line like that, in your Pyndent .pyn file:

#!/usr/bin/pyndent <options> <filename>
(Python code here)

At the same time, a Python programmer, even if using Pyndent to pre-process his code, would like to use the hashbang to call the Python interpreter, in his final .py source.
How to achieve it?

It can be achieved by using a simple rule of hashbangs co-existence: Pyndent's hashbang precedes Python's one, in .pyn meta-sources, as well as Python's hashbang will precede Pyndent's one in .py sources: the swap will be done by Pyndent this way:

(Pyndent meta-source: example.pyn)

#!/usr/bin/pyndent -o example.py
#!/usr/bin/env python3
#delim {{ }}

import sys

def example():
{{
print("example!")
}}

example()

The above meta-source will be processed this way by Pyndent:

(Python source: example.py)

#!/usr/bin/env python3
#!/usr/bin/pyndent -o example.py
#delim {{ }}

import sys

def example():
#{{
	print("example!")
#}}

example()

Python won't complain, if finding the #!/usr/bin/pyndent -o example.py as second line: it will manage it as a normal comment. The result will be 100% Python code (with Pyndent source hashbang, directive(s), and (custom) delimiters retained).
This way, if anyone will need to --restore .py code to the original meta-code, Pyndent will be able to read its own original hashbang (if present, it will re-swap it with Python's one), the directives (#delim in our case) are untouched and don't need to be "restored", all delimiters are still there, even commented (#{{, #}}) and can be easily brought back: {{, }} (also considering #delim contents, if in doubt).


Syntax scheme

pyndent [option]... <meta>.pyn [<source>.py]
for [options]: [-o][-e][-x][-s][-f][-i][-v]

pyndent <source>.py [option]... [<meta>.pyn]
for [options]: [-r][-f][-i][-v]

pyndent [option]...
for [options]: [-h][-V]

pyndent
is equivalent to pyndent --help

If no option is given, Pyndent default behavior is to read a meta-source .pyn file, translate it into a re-indented Python source code writing it to <stdout>. In this case the meta-source .pyn file is a mandatory argument.


Options: released and planned

Simple Extended Description
-e --exec
--execute
tells Pyndent to launch Python, to execute the processed code (from <file>.py or <stdout>), if the pre-processing ended correctly (RC = 0).
By default, the code is written to <stdout>, so it's then read from <stdin> by Python (unless -o or -x are used)
-o --outfile asks Pyndent to write the output to a <filename>.py instead of <stdout>.
<filename> is optional: if not given, the meta-source one is used
-x --execout
--execute-output
combo switch implying -e and -o: everything valid for -e and -o is valid for -x too
(-x and -e/-o are of course mutually exclusive)
-f --force allows to silently overwrite the output file if already present: if not given and Pyndent finds the output file when going to write it, an error is emitted and the original target file is not overwritten
Force is an option to -o, -x and -r
-q --quiet Suppress any output to <stdout> only (file output remains, if requested)
-i --interactive prompts to replace an output target file, if found, instead of exit in error
-s --strip
--strip-delims
strips the delimiters away from the final Python code (AKA avoid to write them out at all), as well as every Pyndent element (like #delim or hashbang swapping), producing 100% pure Python source without any Pyndent meta-source element into.
-r --restore asks Pyndent to restore (de-process) a meta-source from a Python .py source (mandatory), restoring all the Pyndent elements (delimiters, hashbang if present) from commented ones
having Pyndent elements commented in the source .py file is not mandatory if the source correctly executes in Python
(a .pyn file will be written, if using -o switch)
(disables -e and -x option)
-v --verbose will write all --verbose messages to <stderr>, for the asked verbosity level (where -v/--verbose = INFO, -vv = DEBUG, -vvv = TRACE), as well as the produced Python code to <stdout> (unless -o switch is given).
--vl
--verblog
--verbose-logfile
Write a .log for verbose option (use without value for auto-name, default output to <stderr> if this option is not given)
--vp
--verbpfx
--verbose-prefix
0..3
Timestamp prefix for verbose logfile: 0=no prefix (overwrite any existing logfile), 1=YYYYMMDD, 2=YYYYMMDD-HHMM, 3=YYYYMMDD-HHMMSS
-h --help shows simpler usage (-h) or full help (--help)
-V --version shows the current Pyndent version

note: always check the ROADMAP to see which options are available already.


Options in detail

By default, Pyndent reads a meta-source (<filename>.pyn) and writes a full-Python source to <stdout>, nothing else: it only rewrite indentation from scratch.
As it can be customized to perform different tasks, you can use the following switches too:


-e --execute

Tells Pyndent to launch Python, passing it the processed Python code (from file or <stdout>) to process, if the pre-processing ended correctly (RC = 0). By default, the code is written to <stdout>, so it's then read from <stdin> by Python (unless -o or -x are used)

pyndent -e meta.pyn

  • will produce a Python source stream to <stdout>, then launch Python which will read its <stdin> to execute it

pyndent -e meta.pyn -o source.py

  • will produce a source.py file, then launch Python source.py to execute it

pyndent -e meta.pyn -o

  • the same, but will produce meta.py file, then launch Python meta.py to execute it

-o --outfile

Asks Pyndent to write the output to a <filename>.py instead of <stdout>. <filename> is optional: if not given, the meta-source one is used

pyndent -o source.py meta.pyn

  • will read meta.pyn and write source.py file, instead of just output the processed results to <stdout>

pyndent meta.pyn -o source.py

  • the same, with more "natural" syntax: input first, then output file as last

pyndent -o meta.pyn

  • will read meta.pyn and write meta.py file: <metasource> filename is taken from meta.pyn input

pyndent meta.pyn -o

  • the same, even if less intuitive but it works too

-x --execute-output

Combo switch implying -e and -o

pyndent -x meta.pyn

  • will be internally translated into pyndent -e -o meta.pyn, resulting in meta.py file on disk, then Python interpreter executing it

pyndent -x source.py meta.pyn

  • will be internally translated into pyndent -e -o source.py meta.pyn, resulting in source.py file written to disk, then Python interpreter is called to execute the created file
be warned that, despite obvious, -x and -e/-o options are mutually exclusive

-f --force

Asks Pyndent to overwrite the output without asking, if a target file is found

pyndent -o source.py meta.pyn or pyndent meta.pyn -o source.py

  • will exit in error if <source>.py is present already
    pyndent -o source.py meta.pyn -f or pyndent meta.pyn -f -o source.py
    pyndent -x source.py meta.pyn -f or pyndent meta.pyn -f -x source.py
  • will overwrite <source>.py if found

pyndent meta.pyn -o

  • will exit in error if <meta>.py is present already pyndent meta.pyn -o -f
    pyndent -x meta.pyn -f
  • will overwrite <meta>.py if found

pyndent -r source.py meta.pyn -f

  • will overwrite <source>.py if found

pyndent meta.pyn -r -f

  • will overwrite <meta>.py if found

pyndent meta.pyn -o source.py

  • the same, with more "natural" syntax: input first, then output file as last

pyndent -o meta.pyn

  • will read meta.pyn and write meta.py file: <metasource> filename is taken from meta.pyn input

pyndent meta.pyn -o

  • the same, even if less intuitive but it works too

| [-q](#optdet_q) | [--quiet](#optdet_q) | Suppress any output to \ **only** (file output remains, if requested) |

-q --quiet

Suppress any output to <stdout> only (file output remains, if requested)

pyndent -q meta.pyn

  • will process meta.pyn file without any processing output to <stdout> (useful for scripts which needs the RC only or to validate the meta-source). If any error will arise, it will be shown, tho, and RC will be set accordingly.

-s --strip

Asks Pyndent to strip the delimiters away from the final Python code (AKA avoid to write them out at all), as well as every Pyndent element (like #delim or hashbang swapping), producing 100% pure Python source without any Pyndent meta-source element into.

pyndent -s meta.pyn

  • will write the resulting Python code to <stdout> without any of the Pyndent elements

pyndent -s -o meta.pyn

  • the same, but the resulting code will be written into meta.py file

-r --restore

Asks Pyndent to reverse-process (de-process) a Python .py source into a Pyndent meta-source, restoring all the Pyndent elements (delimiters, hashbang if present) from commented ones (a .pyn file will be written, if using -o switch).
If you need to --restore a pure Python source (or even a Pyndent --strip one), which totally lacks Pyndent elements (even commented), you could give a #delim <start_delimiter> <stop_delimiter> directive too, inline or into Python source code (starting from 2nd line or below), if you want to override the defaults.

pyndent -r source.py

  • will write a Pyndent meta-source to <stdout>, honoring the inline #delim if given, then seasrching for an embedded #delim directive in the source file, if lacking the inline one, then falling back to defaults if no directive is found (precedence rule: inline -> embedded -> defaults).

pyndent -r #delim {{ }} source.py

  • the same, but Pyndent assumes there are no pre-existing (commented) delimiters to restore (just skipping existing ones, leaving them commented out): it will add the given delimiters instead.

-v --verbose

Will write all --verbose messages to <stderr>, for the asked verbosity level (where --v1 = INFO, --v2 = DEBUG), as well as the produced Python code to <stdout> (unless -o switch is given).

pyndent -v meta.pyn

  • will read meta.pyn, write processed code to <stdout>, and INFO level messages to <stderr>

pyndent --verbose meta.pyn

  • the same: -v, and --verbose are aliases for INFO messages level

pyndent -v -o source.py meta.pyn

  • will read meta.pyn and write source.py file, and INFO level messages to <stderr>. INFO messages are just the main ones, confirming the tool is working.

pyndent -v meta.pyn -o source.py

  • the same, with more "natural" syntax

pyndent -vv -o source.py meta.pyn

  • the same, but with a lot more details: INFO and DEBUG level messages will be written to <stderr>. DEBUG messages gives much more informations on options, input and output file, together with processing details on read meta-code.

pyndent -vvv -o source.py meta.pyn

  • the same, but with even more details: INFO, DEBUG, and TRACE level messages will be written to <stderr>. TRACE adds to DEBUG level some Pyndent's inner states, warnings and errors details.

--vl --verblog --verbose-logfile

Write a .log for verbose option (use without value for auto-name, default output to <stderr> if this option is not given)

pyndent meta.pyn -v --vl

  • will read meta.pyn and write meta.log file with INFO level informations, the processed output will be sent to <stdout>

pyndent meta.pyn -o -vv --vl

  • will read meta.pyn and write meta.log file with DEBUG level informations, the processed output will be sent to meta.py file.

pyndent meta.pyn -o source.py -vvv --vl processing.log

  • will read meta.pyn and write processing.log file with TRACE level informations, the processed output will be sent to source.py file.

--vp --verbpfx --verbose-prefix [<0..3>]

Timestamp prefix for verbose logfile: 0=no prefix (overwrite any existing logfile), 1=<YYYYMMDD>, 2=<YYYYMMDD-hhmm>, 3=<YYYYMMDD-hhmmss>

pyndent meta.pyn -v --vl

  • will read meta.pyn and write meta.log file with INFO level informations, the processed output will be sent to <stdout>. meta.log filename won't be timestamped: if you'll repeat the command, it will be silently overwritten (good for non-progressive logfiles).

pyndent meta.pyn -o -vv --vl --vp 1

  • will read meta.pyn and write <YYYYMMDD>_meta.log file with DEBUG level informations, the processed output will be sent to meta.py file. Logfile timestamp will persist for the entire day: during that time every rewrite to meta.log file will silently overwrite it.

pyndent meta.pyn -o source.py -vvv --vl processing.log --vp 2

  • will read meta.pyn and write <YYYYMMDD-hhmm>processing.log file with TRACE level informations, the processed output will be sent to source.py file. Logfile timestamp will persist for a minute: during that time every rewrite to meta.log file will silently overwrite it.

pyndent meta.pyn -o source.py -vvv --vl fast.log --vp 3

  • will read meta.pyn and write <YYYYMMDD-hhmmss>fast.log file with TRACE level informations, the processed output will be sent to source.py file. Logfile timestamp will persist for a second: during that time every rewrite to meta.log file will silently overwrite it.

-V --version

Shows the current Pyndent version.


-h --help

Shows simpler usage (-h) or full help (--help).


Obsoletes

-d --dryrun

Asks Pyndent to simulate the reindentation process, also doing all the needed checks, showing the results to <stdout>, not writing the output .py file

pyndent -d meta.pyn

  • will write the results to <stdout> but won't write to file

no more needed as that's now Pyndent default behavior


Document History

20250925 original
20250925 fixed typos and examples added (Aria@DeepSeek)
20251102 markdown translation