..
  This file is part of GNU TALER.
  Copyright (C) 2014-2024 Taler Systems SA

  TALER is free software; you can redistribute it and/or modify it under the
  terms of the GNU Affero General Public License as published by the Free Software
  Foundation; either version 2.1, or (at your option) any later version.

  TALER is distributed in the hope that it will be useful, but WITHOUT ANY
  WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
  A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License along with
  TALER; see the file COPYING.  If not, see <http://www.gnu.org/licenses/>

  @author Florian Dold
  @author Marcello Stanisci
  @author Christian Grothoff

.. target audience: operator, developer

.. _libeufin-nexus:

Nexus Manual
############

LibEuFin Nexus is an EBICS facilitator.  It offers a command line interface to
setup EBICS access, download banking records, and submit payments.  Today, the
LibEuFin implementation supports EBICS 2.5 and 3.0 and has been tested with
the Postfinance (CHF). Please note that banks tend to have their own dialects
of finance messages and thus other retail banks may or may not work.  Contact
us if you need support for another bank or core banking protocol.

Future versions will offer a Web API to allow Taler exchanges to talk to their
banks.

In this manual, we explain how to setup an EBICS subscriber.  We assume that
the bank has already granted EBICS access to the subscriber.

.. contents:: Table of Contents
  :local:


Installing Nexus
================

The following section was tested on an *OpenJDK 17* environment.


Installing the libeufin-nexus binary packages on Debian
-------------------------------------------------------

.. include:: ../frags/installing-debian.rst

.. include:: ../frags/apt-install-libeufin-nexus.rst


Installing the libeufin-nexus binary packages on Ubuntu
-------------------------------------------------------

.. include:: ../frags/installing-ubuntu.rst

.. include:: ../frags/apt-install-libeufin-nexus.rst


Building from source
--------------------

Nexus belongs to the LibEuFin project, and can be downloaded via Git:

.. code-block:: console

  $ git clone git://git.taler.net/libeufin

Note that Kotlin and Gradle should already work on the host system.

Navigate into the *libeufin* local repository, and from top-level run:

.. code-block:: console

  $ ./bootstrap
  $ ./configure --prefix=$PREFIX
  $ make install

If the previous steps succeeded, the ``libeufin-nexus`` command should
be found in the $PATH.

.. _ebics-setup:

Database setup
==============

The configuration file must include a connection string that
tells Nexus how it should connect to the database. The default
is:

.. code-block:: ini

  [libeufin-nexusdb-postgres]
  config = postgres:///libeufin

You must make sure that this database exists and is accessible to the user
running Nexus before continuing.  Then, the Nexus database schema must be
created (or updated) to the current Nexus version using the following command:

.. code-block:: console

  $ libeufin-nexus-dbinit -c "$CONFIG_FILE"

where ``$CONFIG_FILE`` is again the path to a configuration that contains at
least the above ``[libeufin-nexusdb-postgres]`` section.

Setting up the EBICS subscriber
===============================

.. include:: ../frags/nexus-ebics-setup.rst

.. _sending-payments:

Sending payments
================

Sending payments must follow a successful `EBICS subscriber setup
<ebics-setup>`_, where the bank obtained the subscriber keys, and the
subscriber accepted the bank keys. Furthermore, the database has to
be set up. Right now, some other process must queue payments to be
submitted to the database, so this section only discusses how such
queued payments will be executed.

The responsible subcommand for sending payments is ``ebics-submit``, and its
configuration is a **superset** of core-config_.  On top of that, it expects
the database connection string and *optionally* a frequency at which it will
check for submittable payments in the database.

The frequency **may** be specified as

.. code-block:: ini

  [nexus-submit]
  FREQUENCY = 5m

The supported time units are ``s`` (seconds), ``m`` (minutes), ``h`` (hours).


For testing
-----------

The ``ebics-submit`` subcommand is **not** suitable to send arbitrary
payments, but rather to submit initiated payments that are already queued for
submission in the Nexus database.

Such queued payments may originate from bounces of incoming payments with a
subject that would be invalid for a Taler withdrawal, or from a Taler exchange
that needs to send a payment to a merchant.

For testing purposes it is possible to manually initiate a payment using the
``initiate-payment`` subcommand.
The following invocation pays 1 CHF to the CH987654321 with an "Hello, Nexus!"
subject:

.. code-block:: console

  $ libeufin-nexus initiate-payment -c "$CONFIG_FILE" --subject "Hello, Nexus!" "payto://iban/CH987654321?receiver-name=Name&amount=CHF:1"

``$CONFIG_FILE`` is the path to Nexus' configuration file and it does not need
the FREQUENCY value.  If the previous command worked, now the database should
contain a new row in the ``initiated_outgoing_transactions`` table, with an
``unsubmitted`` submission state.

The following command would now pick such an unsubmitted initiated payment and
send it to the bank.


.. code-block:: console

  $ libeufin-nexus ebics-submit -c "$CONFIG_FILE" --transient

The ``--transient`` flag instructs the software to perform only one pass on
the database, submit what is submittable, and return.

The ``--debug-ebics $SAVEDIR`` flag will cause EBICS transactions steps and payloads to be stored in ``$SAVEDIR/$YYYY-MM-DD`` where ``$YYYY-MM-DD`` represents the date when the upload took place. This is mostly useful for debugging.

For production
--------------

The ``ebics-submit`` subcommand can run in fixed frequency, or transient mode.

The fixed frequency mode causes the command to check the database every time the
frequency period expires.  To activate this mode, and -- for example -- set a frequency
of five minutes, set the configuration value ``FREQUENCY`` to ``5m``.  Assuming that
``FREQUENCY`` is set as described above, the following invocation would
make ``ebics-submit`` check the database every five minutes, and submit any initiated
payment according to their submission state.

.. code-block:: console

  $ libeufin-nexus ebics-submit -c "$CONFIG_FILE"

Transient mode causes ``ebics-submit`` to check the database only once, submit any
initiated payment according to their submission state, and return.

.. code-block:: console

  $ libeufin-nexus ebics-submit -c "$CONFIG_FILE"


.. _receive-transaction-data:

Downloading payment records
===========================

Downloading payments must follow a successful `EBICS subscriber setup
<ebics-setup>`_, where the bank obtained the subscriber keys, and the
subscriber accepted the bank keys. Furthermore, the database has to
be set up.

The responsible subcommand for fetching and registering payments is 
``ebics-fetch``, and its configuration is a **superset** of core-config_. On 
top of that, it expects the database connection string and *optionally* a 
frequency and a checkpoint time of day.

For the best user experience, new transactions should be registered as soon as 
they are booked. This is especially important for instant transactions, which 
reach final status in ~10 seconds. On the other hand, EBICS document fetches 
are slow and expensive and should not be performed more than necessary.

``ebics-fetch`` will run a periodical fetch of pending documents (documents 
that have never been fetched before). If the bank server supports it, it will 
also listen to real-time document availability notifications and fetch 
documents as soon as they are announced. This enables instant registration of 
instant transactions. If your bank server does not support real-time 
notifications, it is recommended that you fetch more often; otherwise, you can 
reduce the fetch frequency.

Only fetching pending documents is not always reliable, if other EBICS clients 
use the same connection settings as ``libeufin-nexus``, they can consume documents before they are ingested. In case of bank downtime, the status of 
documents can be lost. This is why ``ebics-fetch`` will run a checkpoint every 
day to download all documents generated since the last checkpoint.

If the bank supports real-time notifications, you can expect transactions to be registered as soon as they are booked: ~10s for instant transactions and 24 to 72h for other transactions. Otherwise, expect a latency corresponding to the fetch frequency for instant transactions. Thanks to checkpointing, latency should not exceed one day.

The frequency **may** be specified as

.. code-block:: ini

  [nexus-fetch]
  FREQUENCY = 5m

The supported time units are ``s`` (seconds), ``m`` (minutes), ``h`` (hours).

The checkpoint time of day **may** be specified as

.. code-block:: ini

  [nexus-fetch]
  CHECKPOINT_TIME_OF_DAY = 19:00

Assuming that ``$CONFIG_FILE`` contains all required options, the following 
command would download any unseen EBICS files:

.. code-block:: console

  $ libeufin-nexus ebics-fetch -c "$CONFIG_FILE" --transient

The ``ebics-fetch`` subcommand will always cause libeufin-nexus to download
then parse EBICS files and register their content statements into its local
database.

The ``--transient`` flag makes the command download and import EBICS files only 
once and return immediately afterwards.  Without the flag, Nexus
will download at the frequency specified in the configuration or when it receive real time notifications.

The ``--debug-ebics $SAVEDIR`` flag will cause EBICS transactions steps and payloads to be stored in ``$SAVEDIR/$YYYY-MM-DD`` where ``$YYYY-MM-DD`` represents the date when the download took place. This is mostly useful for debugging.


For Testing
-----------

If the bank account history has transactions that are useful for testing, it is
possible to re-download them via the ``--pinned-start`` argument.

The following command redownloads transactions *from* the pinned start until
the present:

.. code-block:: console

  $ libeufin-nexus ebics-fetch -c "$CONFIG_FILE" --pinned-start 2023-11-01

.. note::

  If the Nexus database already contains the bank account history, you might need to **reset** the database in order to have the re-downloaded transactions be properly registered.

  Be careful when resetting the database: resetting the wrong database might cause DATA LOSS!

  Use the ``IGNORE_BOUNCES_BEFORE`` configuration to prevent malformed transactions from being bounced a second time when they are registered again.

Manual EBICS import & export
============================

If your bank does not support the EBICS protocol, but has an interface for importing and exporting ISO20022 files, you can run the deployment manually.

Sending payments
----------------

To obtain PAIN files for pending initiated outgoing transactions in a `payments.zip` file, you can run :

.. code-block:: console

  $ libeufin-nexus manual export -c "$CONFIG_FILE" payments.zip

Each file in the ZIP file corresponds to a single batch of transactions. The command prints the details of each batch and their internal transactions. Files are generated as if they would be uploaded using the EBICS protocol.

As we have no way of knowing whether the upload was successful, you need to update the batch status manually. Until you tell ``libeufin-nexus`` that a batch has been uploaded successfully, it will re-export it next time.

If the batch has been uploaded successfully, you can run this command:

.. code-block:: console

  $ libeufin-nexus manual status -c "$CONFIG_FILE" batch $BATCH_ID pending

Alternatively, if the error is transient, for example if the bank's user interface is under maintenance, you can set a temporary error accompanied by a message with:

.. code-block:: console

  $ libeufin-nexus manual status -c "$CONFIG_FILE" batch $BATCH_ID transient_failure "In maintenance until Wednesday morning"

If the error is permanent, for example if the file is malformed and your bank will never accept it, you can define a permanent error with:

.. code-block:: console

  $ libeufin-nexus manual status -c "$CONFIG_FILE" batch $BATCH_ID permanent_failure "Unuspported transaction kind"

It can also happen that a batch is partially successful, the batch is uploaded but some transactions failed, for example because a creditor's IBAN is incorrect. You can mark a batch as uploaded even though some transactions have failed with:

.. code-block:: console

  $ libeufin-nexus manual status -c "$CONFIG_FILE" batch $BATCH_ID pending
  $ libeufin-nexus manual status -c "$CONFIG_FILE" tx $TX_ID permanent_failure "Malformed IBAN"

Successful transactions should be announced in CAMT files, so you shouldn't have to mark them as successful manually, but you still can with:

.. code-block:: console

  $ libeufin-nexus manual status -c "$CONFIG_FILE" tx $TX_ID success

Import payment record
---------------------

To import CAMT files, you can run:

.. code-block:: console

  $ libeufin-nexus manual import -c "$CONFIG_FILE" camt53.xml

Files will be processed as if they had been downloaded using the EBICS protocol.