DD 41: Wallet Balance and Amount Definitions
############################################
Summary
=======
This design document discusses terminology and concepts used in the wallet
for balances and amounts.
Motivation
==========
There are many different types of balances and amounts, and they need
to have a clear definition.
Furthermore, the user in some situations needs to know/decide whether
an amount that the user chooses includes fees or not.
Proposed Solution
=================
Amounts
-------
* "effective amount": An effective amount always represents the direct effect on the
wallet's balance of the same currency.
* "raw amount": The raw amount always refers to the amount with fees applied.
The exact interpretation of that depends on the transaction type.
* "instructed amount": An instructed amount always refers to the amount that
a user has explicitly specified as an input. It is not directly a property
of transactions, but might be added as metadata to transactions for
informational purposes. How the instructed amount is interpreted
differs based on the "instructed amount mode" that is specified
together with the amount. By default, the instructed amount
is assumed to translate to the raw amount of the corresponding transaction.
* "counter-party effective amount": An amount that **estimates** the effect
of the transaction of the balance (either wallet or bank account) of the other
party. This is usually a conservative estimate, i.e. when sending money,
this is the lower bound for the funds that the other party will obtain
*after* fees.
Transaction types initialized by the wallet
-------------------------------------------
MANUAL_WITHDRAW
raw amount is the amount that need to be added into the exchange account,
this is not the same as the amount leaving the user account since their own
account may charge some fee that can be taken into account
``coins`` = select-coin-for-operation(credit, mode, instructed_amount)
if instructed_amount mode = raw
``raw_amount`` = instructed_amount
``effective_amount`` = instructed_amount - coins.withdrawal_fee
if instructed_amount mode = effective
``raw_amount`` = instructed_amount + coins.withdrawal_fee
``effective_amount`` = instructed_amount
DEPOSIT
raw amount is the amount leaving the exchange account without the wire fee,
this should be what the user see as an increase in the user bank account unless
there are some extra fee not seen from the exchange
``coins`` = select-coin-for-operation(debit, mode, instructed_amount)
if instructed_amount mode = raw
``raw_amount`` = instructed_amount
``effective_amount`` = instructed_amount + coins.deposit_fee + coins.refresh_fee + wire.transfer_fee
if instructed_amount mode = effective
``raw_amount`` = instructed_amount - coins.deposit_fee - coins.refresh_fee - wire.transfer_fee
``effective_amount`` = instructed_amount
PULL CREDIT (creating an invoice)
raw amount is the amount in the exchange that counter-party need will pay (purse_value),
this is exactly the same raw amount of the PULL DEBIT operation (paying the invoice)
counter-party amount is an aprox amount of the other wallet assuming the same coin selection
algorithm
``coins`` = select-coin-for-operation(credit, mode, instructed_amount)
if instructed_amount mode = raw
``raw_amount`` = instructed_amount
``effective_amount`` = instructed_amount - coins.withdrawal_fee - purse_fee
if instructed_amount mode = effective
``raw_amount`` = instructed_amount + coins.withdrawal_fee + purse_fee
``effective_amount`` = instructed_amount
if instructed_amount mode = counter-party
``raw_amount`` = instructed_amount - coins.counter-party_deposit_fee
``effective_amount`` = instructed_amount - coins.counter-party_deposit_fee - coins.withdrawal_fee - purse_fee
``counter-party_raw_amount`` = raw_amount
``counter-party_effective_amount`` = raw_amount + coins.counter-party_deposit_fee
.. note::
counter-party_effective_amount is an estimation since refresh fee is not included.
Refresh fee can't be calculated because depends on the coins available in the wallet
of the counter-party
.. note::
coins.counter-party_deposit_fee is the minimum deposit_fee that can be calculated for the
given exchange. Counter-party may pay more if it have different preferences doing the coin
selection.
PUSH DEBIT (creating a transfer)
raw amount is the amount in the exchange that counter-party need be able to withdraw (purse_value),
this is exactly the same raw amount of the PUSH CREDIT operation (getting the transfer)
counter-party amount is an aprox amount of the other wallet assuming the same coin selection
algorithm
``coins`` = select-coin-for-operation(debit, mode, instructed_amount)
if instructed_amount mode = raw
``raw_amount`` = instructed_amount
``effective_amount`` = instructed_amount + coins.deposit_fee + purse_fee
if instructed_amount mode = effective
``raw_amount`` = instructed_amount - coins.deposit_fee - purse_fee
``effective_amount`` = instructed_amount
if instructed_amount mode = counter-party
``raw_amount`` = instructed_amount + coins.counter-party_withdraw_fee
``effective_amount`` = instructed_amount - coins.counter-party_withdraw_fee - coins.withdrawal_fee - purse_fee
``counter-party_raw_amount`` = raw_amount
``counter-party_effective_amount`` = raw_amount - coins.counter-party_withdraw_fee
.. note::
coins.counter-party_withdraw_fee is the minimum withdraw_fee that can be calculated for the
given exchange. Counter-party may pay more if it have different preferences doing the coin
selection.
Transaction types completed by the wallet
-------------------------------------------
Next transaction types are not initiated in the wallet so instructed amount is not defined by the wallet user.
We need to calculate effective_amount to check if the wallet is able to perform the operation or the user accept
the fees.
BANK_WITHDRAW
raw amount is that reached the bank account of the exchange,
this is not the same as the amount leaving the user account since their own
account may charge some fee that can be taken into account
``coins`` = select-coin-for-operation(credit, mode, instructed_amount)
if instructed_amount mode = raw
``raw_amount`` = instructed_amount
``effective_amount`` = instructed_amount - coins.withdrawal_fee
if instructed_amount mode = effective
``raw_amount`` = instructed_amount + coins.withdrawal_fee
``effective_amount`` = instructed_amount
.. note ::
how much wire_fee the merchant is willing to pay
``merchant_wire_fee`` = min(wire.transfer_fee / contractTerms.amortization_factor, contractTerms.max_wire_fee)
``merchant_deposit_fee`` = min(contractTerms.max_fee, contract_wire_fee)
PAYMENT
raw amount is the amount the merchant should get if is not doing aggregated transaction.
``instructed_amount`` = contractTerms.amount
``coins`` = select-coin-for-operation(debit, mode, raw_amount)
``raw_amount`` = instructed_amount - merchant_deposit_fee
``effective_amount`` = instructed_amount + coins.deposit_fee + coins.refresh_fee + (wire.transfer_fee - merchant_wire_fee)
.. note::
The current coin-selection algorithm the order_price is neither raw_amount nor effective_amount.
We can calculate the raw_amount of the payment as (contractTerms.amount - max_merchant_fee) and then this
operation becomes equivalent than a deposit (in terms of fee calculation).
PUSH CREDIT (getting the transfer)
raw amount is the purse_value in the exchange to be withdrawn
``instructed_amount`` = p2pContract.amount
``coins`` = select-coin-for-operation(credit, mode, raw_amount)
``raw_amount`` = instructed_amount
``effective_amount`` = instructed_amount - coins.withdrawal_fee
.. note::
In the case that the withdrawal_fee of the coin selection for the push-credit amount
is higher than the wire_fee of the exchange, can the wallet ask the exchange to make
a wire transfer of the purse instead of proceeding?
PULL DEBIT (paying an invoice)
raw amount is the net value of the invoice without fees
``instructed_amount`` = p2pContract.amount
``coins`` = select-coin-for-operation(debit, mode, raw_amount)
``raw_amount`` = instructed_amount
``effective_amount`` = instructed_amount + coins.deposit_fee + coins.refresh_fee + wire.transfer_fee
REFUND
raw amount is the amount that the merchant refunded
``instructed_amount`` = refund.amount
``raw_amount`` = instructed_amount
``effective_amount`` = instructed_amount - refund_fee - refresh_fee
.. note::
There may be the case that the merchant should refund all the value of the purchase
and that may include paying for the refund_fee.
Is there a way that the merchant can initiate a refund of purchase + refund_fee so
the wallet will get the same effective_amount?
REWARD
raw amount is the amount that the merchant send as reward
``instructed_amount`` = reward.amount
``raw_amount`` = instructed_amount + withdrawal_fee
``effective_amount`` = instructed_amount
.. note::
We should not show fee for rewards in the wallet since the merchant is the one choosing
the exchange and we can assume that those rewards are paid by the merchant.
So the wallet only care about the effective.
Coin selection algorithm
------------------------
Is an internal optimization algorithm that will choose coins given a denomination list or current coin list
until amount is reached. The coins selected to minimize the fee spent.
``select-coin-for-operation`` will receive 3 parameters:
* operation: define if the coins are selected from wallet database or from denomination list. Possible values are:
- credit: reduce withdrawal fee, use exchange denomination list
- debit: reduce deposit fee, use database denomination and current coin count
* amount: how much value the coins need to sum up
* mode: the interpretation of the amount parameter
- net: the amount does not include the operation fee
- gross: the amount include the operation fee
If the operation is debit and with the current coins there is no way to reach the amount then
1. an optimized withdrawal operation can be suggested (list of denominations)
2. an optimized refresh operation can be suggested (amount gap, coin to be refreshed and list of denominations to withdraw)
.. note::
``select-coin-for-operation`` must be predictable (select the same coins for the same parameters) and if
the difference between two amounts are the fee for a given operation:
operation: credit
withdrawal_fee = amount1 - amount2
then the wallet should select the same coins using the correct mode
select-coin(withdraw, raw, amount1) == select-coin(withdraw, effective, amount2)
Instructed Amount Modes
-----------------------
The interpretation and possible choices of the instructed amount mode
depends on which transaction is initiated.
For withdrawal:
* ``raw-mode`` (default): instructed amount is what is subtracted from the reserve balance (i.e. it's the raw amount)
* ``effective-mode``: instructed amount is what the user wants to have as material balance in the wallet
FIXME(dold): However, that does not really cover the user case where the merchant charges fees and the user has to pay for that.
So in theory we could have a mode that withdraws enough to pay for some particular claimed order, but IMHO that's overkill.
For deposits (where there is no contract that already specifies an amount):
* ``max-mode``: The instructed amount is ignored (can be zero in the request), and all coins are spent (note that the calculation should be made available when the user is asked to specify an amount when using a template)
* ``raw-mode`` (default): The instructed amount is what will be paid to the "merchant" (or the customer's bank account), ignoring wire fees
* ``effective-mode``: The instructed amount is how the wallet's balance will be affected. Difficult to compute accurately because refresh is involved. Note that the calculation should ideally again be made available when the user is asked to specify an amount when using a template.
For peer-push-debit:
* ``raw-mode`` (default): The instructed amount is what will be paid, deposit fees are covered by the sender, withdrawal fees from the reserve by the receiver
* ``effective-mode``: Instructed amount is the effect on the material balance. Difficult to compute accurately because refresh is involved.
* ``counter-party-effective-mode``: Instructed amount is the effect on the counterparty's wallet balance. Difficult to compute accurately because coin selection by receiver may not match our expectations.
FIXME(dold): Should we also have a mode where withdrawal fees are covered by the side that does peer-push-debit? However, that assumes the other party has the same withdrawal coin selection algorithm. FIXME(grothoff): isn't this the counterparty-effective-mode you described above, and that would seem to exactly have this issue?
For peer-pull-credit:
* ``raw-mode`` (default): Amount that the other party has to put in the reserve. The payer has to pay any deposit fees on top. The receiver balance is increased by the specified amount minus any withdraw fees.
* ``effective-mode``: Amount by which the initiator's balance is increased. Difficult to compute as the receiver has to simulate different coin selections and their effect on withdraw fees to arrive at the minimum total amount that must be deposited into the reserve.
Illustrative Example
--------------------
To explain the differences between raw, effective and instructed amounts, consider the following scenario: Alice wants to send money
to Bob via a P2P push payment.
Example 1:
* Alice starts a withdrawal of ``KUDOS:10`` from her bank's web interface into her Taler
wallet. The instructed amount is ``KUDOS:10`` and (by default for bank-integrated withdrawals),
the mode is ``raw-mode``. After fees, ``KUDOS:9.8`` arrive in her Taler wallet.
Example 3:
* Alice wants to pay for a ``KUDOS:10`` monthly magazine subscription. Her Taler wallet is empty though.
* She starts withdrawal through her Android wallet app, where she selects ``KUDOS:10`` as the instructed
amount with ``mode=effective-mode``. This translates to ``amountEffective=KUDOS:10`` and ``amountRaw=KUDOS:10.10``.
* Alice is redirected to her banking app where she transfers ``KUDOS:10.10`` to the exchange.
* Her Taler wallet balance will be ``KUDOS:10.10`` after the withdrawal completes.
Note that on the amount she chooses and the fees / denom structure of the exchange, the ``amountEffective`` might be *higher*
than the instructed amount.
FIXME(dold): That flow does not work if withdrawal starts in the bank. Maybe there needs to be a mechanism
where the wallet tells the bank the adjusted amount that needs to be transferred? That would be a new
feature in the bank integration API.
Example 4:
* Alice has ``KUDOS:10`` in her wallet.
* Alice wants to initiate a peer-push payment with ``amountInstructed=KUDOS:8``
and ``mode=effective-mode``. That means that after the payment, she expects
exactly ``KUDOS:2`` to remain in her wallet.
* Due to the fee configuration, her wallet computes ``amountRaw=KUDOS:7.5`` and ``amountEffective=KUDOS:7.8``.
The effective amount in this case does **not** equal the instructed amount, despite the ``mode=effective-mode``.
That's because there no amount that can be spend so that the spend amount with resulting refresh
fees equal ``KUDOS:8``.
* Alice confirms the peer-push payment initiation, and exactly ``KUDOS:7.5`` are credited
to the purse that her wallet creates.
* Bob merges the purse into his reserve. Bob's wallet automatically withdraws
from the reserve, and his wallet balance increases by ``KUDOS:7.1``, since
withdrawal fees are deducted.
Balances
--------
The following types of balances are defined:
- ``available``: Balance that the wallet believes will certainly be available
for spending, modulo any failures of the exchange or double spending issues.
This includes available coins *not* allocated to any
spending/refresh/... operation. Pending withdrawals are *not* counted
towards this balance, because they are not certain to succeed.
Pending refreshes *are* counted towards this balance.
This balance type is nice to show to the user, because it does not
temporarily decrease after payment when we are waiting for refreshes
- ``material``: Balance that the wallet believes it could spend *right now*,
without waiting for any operations to complete.
This balance type is important when showing "insufficient balance" error messages.
- ``age-acceptable``: Subset of the material balance that can be spent
with age restrictions applied.
- ``merchant-acceptable``: Subset of the material balance that can be spent with a particular
merchant (restricted via min age, exchange, auditor, wire_method).
- ``merchant-depositable``: Subset of the merchant-acceptable balance that the merchant
can accept via their supported wire methods.
Balance Mismatch
----------------
The wallet uses the following terminology when an operation can't succeed
because the balance is too low, even though the instructed amount
- "fee-gap-estimate": Additional (material) balance that the wallet estimates it
still needs for the operation to succeed.
- This value is an estimated, because newly withdrawn coins might have different fees.
- This value is specified *per exchange*, because each exchange has different fees.
FIXME(dold): Should we specify an upper-bound fee-gap-estimate to simplify it for the UIs?
Discussion / Q&A
================
(This should be filled in with results from discussions on mailing lists / personal communication.)