pysupported.rst 38.8 KB
Newer Older
dugupeiwen's avatar
dugupeiwen committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
.. _pysupported:

=========================
Supported Python features
=========================

Apart from the :ref:`pysupported-language` part below, which applies to both
:term:`object mode` and :term:`nopython mode`, this page only lists the
features supported in :term:`nopython mode`.

.. warning::
    Numba behavior differs from Python semantics in some situations.  We
    strongly advise reviewing :ref:`pysemantics` to become familiar with these
    differences.


.. _pysupported-language:

Language
========

Constructs
----------

Numba strives to support as much of the Python language as possible, but
some language features are not available inside Numba-compiled functions.
Below is a quick reference for the support level of Python constructs.


**Supported** constructs:

- conditional branch: ``if .. elif .. else``
- loops: ``while``, ``for .. in``, ``break``, ``continue``
- basic generator: ``yield``
- assertion: ``assert``

**Partially supported** constructs:

- exceptions: ``try .. except``, ``raise``, ``else`` and ``finally``
  (See details in this :ref:`section <pysupported-exception-handling>`)

- context manager:
  ``with`` (only support :ref:`numba.objmode() <with_objmode>`)

- list comprehension (see details in this
  :ref:`section <pysupported-comprehension>`)

**Unsupported** constructs:

- async features: ``async with``, ``async for`` and ``async def``
- class definition: ``class`` (except for :ref:`@jitclass <jitclass>`)
- set, dict and generator comprehensions
- generator delegation: ``yield from``

Functions
---------

Function calls
''''''''''''''

Numba supports function calls using positional and named arguments, as well
as arguments with default values and ``*args`` (note the argument for
``*args`` can only be a tuple, not a list).  Explicit ``**kwargs`` are
not supported.

Function calls to locally defined inner functions are supported as long as
they can be fully inlined.

Functions as arguments
''''''''''''''''''''''

Functions can be passed as argument into another function.  But, they cannot
be returned. For example:

.. code-block:: python

  from numba import jit

  @jit
  def add1(x):
      return x + 1

  @jit
  def bar(fn, x):
      return fn(x)

  @jit
  def foo(x):
      return bar(add1, x)

  # Passing add1 within numba compiled code.
  print(foo(1))
  # Passing add1 into bar from interpreted code
  print(bar(add1, 1))

.. note:: Numba does not handle function objects as real objects.  Once a
          function is assigned to a variable, the variable cannot be
          re-assigned to a different function.


Inner function and closure
'''''''''''''''''''''''''''

Numba now supports inner functions as long as they are non-recursive
and only called locally, but not passed as argument or returned as
result. The use of closure variables (variables defined in outer scopes)
within an inner function is also supported.

Recursive calls
'''''''''''''''

Most recursive call patterns are supported.  The only restriction is that the
recursive callee must have a control-flow path that returns without recursing.
Numba is able to type-infer recursive functions without specifying the function
type signature (which is required in numba 0.28 and earlier).
Recursive calls can even call into a different overload of the function.

.. XXX add reference to NBEP

Generators
----------

Numba supports generator functions and is able to compile them in
:term:`object mode` and :term:`nopython mode`.  The returned generator
can be used both from Numba-compiled code and from regular Python code.

Coroutine features of generators are not supported (i.e. the
:meth:`generator.send`, :meth:`generator.throw`, :meth:`generator.close`
methods).

.. _pysupported-exception-handling:

Exception handling
------------------

``raise`` statement
'''''''''''''''''''

The ``raise`` statement is only supported in the following forms:

* ``raise SomeException``
* ``raise SomeException(<arguments>)``

It is currently unsupported to re-raise an exception created in compiled code.

``try .. except``
'''''''''''''''''

The ``try .. except`` construct is partially supported. The following forms
of are supported:

* the *bare* except that captures all exceptions:

  .. code-block:: python

    try:
        ...
    except:
        ...

* using exactly the ``Exception`` class in the ``except`` clause:

  .. code-block:: python

    try:
      ...
    except Exception:
      ...

  This will match any exception that is a subclass of ``Exception`` as
  expected. Currently, instances of ``Exception`` and it's subclasses are the
  only kind of exception that can be raised in compiled code.

.. warning:: Numba currently masks signals like ``KeyboardInterrupt`` and
  ``SystemExit``. These signaling exceptions are ignored during the execution of
  Numba compiled code. The Python interpreter will handle them as soon as
  the control is returned to it.

Currently, exception objects are not materialized inside compiled functions.
As a result, it is not possible to store an exception object into a user
variable or to re-raise an exception. With this limitation, the only realistic
use-case would look like:

.. code-block:: python

  try:
     do_work()
  except Exception:
     handle_error_case()
     return error_code

``try .. except .. else .. finally``
''''''''''''''''''''''''''''''''''''

The ``else`` block and the ``finally`` block of a ``try .. except`` are
supported:

  .. code-block:: python

    >>> @jit(nopython=True)
    ... def foo():
    ...     try:
    ...         print('main block')
    ...     except Exception:
    ...         print('handler block')
    ...     else:
    ...         print('else block')
    ...     finally:
    ...         print('final block')
    ...
    >>> foo()
    main block
    else block
    final block

The ``try .. finally`` construct without the ``except`` clause is also
supported.

.. _pysupported-builtin-types:

Built-in types
==============

int, bool
---------

Arithmetic operations as well as truth values are supported.

The following attributes and methods are supported:

* ``.conjugate()``
* ``.real``
* ``.imag``

float, complex
--------------

Arithmetic operations as well as truth values are supported.

The following attributes and methods are supported:

* ``.conjugate()``
* ``.real``
* ``.imag``

str
---

Numba supports (Unicode) strings in Python 3.  Strings can be passed into
:term:`nopython mode` as arguments, as well as constructed and returned from
:term:`nopython mode`. As in Python, slices (even of length 1) return a new,
reference counted string.  Optimized code paths for efficiently accessing
single characters may be introduced in the future.

The in-memory representation is the same as was introduced in Python 3.4, with
each string having a tag to indicate whether the string is using a 1, 2, or 4
byte character width in memory.  When strings of different encodings are
combined (as in concatenation), the resulting string automatically uses the
larger character width of the two input strings.  String slices also use the
same character width as the original string, even if the slice could be
represented with a narrower character width.  (These details are invisible to
the user, of course.)

The following constructors, functions, attributes and methods are currently
supported:

* ``str(int)``
* ``len()``
* ``+`` (concatenation of strings)
* ``*`` (repetition of strings)
* ``in``, ``.contains()``
* ``==``, ``<``, ``<=``, ``>``, ``>=`` (comparison)
* ``.capitalize()``
* ``.casefold()``
* ``.center()``
* ``.count()``
* ``.endswith()``
* ``.endswith()``
* ``.expandtabs()``
* ``.find()``
* ``.index()``
* ``.isalnum()``
* ``.isalpha()``
* ``.isdecimal()``
* ``.isdigit()``
* ``.isidentifier()``
* ``.islower()``
* ``.isnumeric()``
* ``.isprintable()``
* ``.isspace()``
* ``.istitle()``
* ``.isupper()``
* ``.join()``
* ``.ljust()``
* ``.lower()``
* ``.lstrip()``
* ``.partition()``
* ``.replace()``
* ``.rfind()``
* ``.rindex()``
* ``.rjust()``
* ``.rpartition()``
* ``.rsplit()``
* ``.rstrip()``
* ``.split()``
* ``.splitlines()``
* ``.startswith()``
* ``.strip()``
* ``.swapcase()``
* ``.title()``
* ``.upper()``
* ``.zfill()``

Regular string literals (e.g. ``"ABC"``) as well as f-strings without format specs
(e.g. ``"ABC_{a+1}"``)
that only use string and integer variables (types with ``str()`` overload)
are supported in :term:`nopython mode`.

Additional operations as well as support for Python 2 strings / Python 3 bytes
will be added in a future version of Numba.  Python 2 Unicode objects will
likely never be supported.

.. warning::
    The performance of some operations is known to be slower than the CPython
    implementation. These include substring search (``in``, ``.contains()``
    and ``find()``) and string creation (like ``.split()``).  Improving the
    string performance is an ongoing task, but the speed of CPython is
    unlikely to be surpassed for basic string operation in isolation.
    Numba is most successfully used for larger algorithms that happen to
    involve strings, where basic string operations are not the bottleneck.


tuple
-----

Tuple support is categorised into two categories based on the contents of a
tuple. The first category is homogeneous tuples, these are tuples where the type
of all the values in the tuple are the same, the second is heterogeneous tuples,
these are tuples where the types of the values are different.

.. note::

    The ``tuple()`` constructor itself is NOT supported.

homogeneous tuples
------------------

An example of a homogeneous tuple:

.. code-block:: python

    homogeneous_tuple = (1, 2, 3, 4)

The following operations are supported on homogeneous tuples:

* Tuple construction.
* Tuple unpacking.
* Comparison between tuples.
* Iteration and indexing.
* Addition (concatenation) between tuples.
* Slicing tuples with a constant slice.
* The index method on tuples.

heterogeneous tuples
--------------------

An example of a heterogeneous tuple:

.. code-block:: python

    heterogeneous_tuple = (1, 2j, 3.0, "a")

The following operations are supported on heterogeneous tuples:

* Comparison between tuples.
* Indexing using an index value that is a compile time constant
  e.g. ``mytuple[7]``, where ``7`` is evidently a constant.
* Iteration over a tuple (requires experimental :func:`literal_unroll` feature,
  see below).

.. warning::
   The following feature (:func:`literal_unroll`) is experimental and was added
   in version 0.47.

To permit iteration over a heterogeneous tuple the special function
:func:`numba.literal_unroll` must be used. This function has no effect other
than to act as a token to permit the use of this feature. Example use:

.. code-block:: python

    from numba import njit, literal_unroll

    @njit
    def foo():
        heterogeneous_tuple = (1, 2j, 3.0, "a")
        for i in literal_unroll(heterogeneous_tuple):
            print(i)

.. warning::
    The following restrictions apply to the use of :func:`literal_unroll`:

    * :func:`literal_unroll` can only be used on tuples and constant lists of
      compile time constants, e.g. ``[1, 2j, 3, "a"]`` and the list not being
      mutated.
    * The only supported use pattern for :func:`literal_unroll` is loop
      iteration.
    * Only one :func:`literal_unroll` call is permitted per loop nest (i.e.
      nested heterogeneous tuple iteration loops are forbidden).
    * The usual type inference/stability rules still apply.

A more involved use of :func:`literal_unroll` might be type specific dispatch,
recall that string and integer literal values are considered their own type,
for example:

.. code-block:: python

    from numba import njit, types, literal_unroll
    from numba.extending import overload

    def dt(x):
        # dummy function to overload
        pass

    @overload(dt, inline='always')
    def ol_dt(li):
        if isinstance(li, types.StringLiteral):
            value = li.literal_value
            if value == "apple":
                def impl(li):
                    return 1
            elif value == "orange":
                def impl(li):
                    return 2
            elif value == "banana":
                def impl(li):
                    return 3
            return impl
        elif isinstance(li, types.IntegerLiteral):
            value = li.literal_value
            if value == 0xca11ab1e:
                def impl(li):
                    # capture the dispatcher literal value
                    return 0x5ca1ab1e + value
                return impl

    @njit
    def foo():
        acc = 0
        for t in literal_unroll(('apple', 'orange', 'banana', 3390155550)):
            acc += dt(t)
        return acc

    print(foo())


list
----


.. warning::
    As of version 0.45.x the internal implementation for the list datatype in
    Numba is changing. Until recently, only a single implementation of the list
    datatype was available, the so-called *reflected-list* (see below).
    However, it was scheduled for deprecation from version 0.44.0 onwards due
    to its limitations. As of version 0.45.0 a new implementation, the
    so-called *typed-list* (see below), is available as an experimental
    feature. For more information, please see: :ref:`deprecation`.

Creating and returning lists from JIT-compiled functions is supported,
as well as all methods and operations.  Lists must be strictly homogeneous:
Numba will reject any list containing objects of different types, even if
the types are compatible (for example, ``[1, 2.5]`` is rejected as it
contains a :class:`int` and a :class:`float`).

For example, to create a list of arrays::

  In [1]: from numba import njit

  In [2]: import numpy as np

  In [3]: @njit
    ...: def foo(x):
    ...:     lst = []
    ...:     for i in range(x):
    ...:         lst.append(np.arange(i))
    ...:     return lst
    ...:

  In [4]: foo(4)
  Out[4]: [array([], dtype=int64), array([0]), array([0, 1]), array([0, 1, 2])]


.. _feature-reflected-list:

List Reflection
'''''''''''''''

In nopython mode, Numba does not operate on Python objects.  ``list`` are
compiled into an internal representation.  Any ``list`` arguments must be
converted into this representation on the way in to nopython mode and their
contained elements must be restored in the original Python objects via a
process called :term:`reflection`.  Reflection is required to maintain the same
semantics as found in regular Python code.  However, the reflection process
can be expensive for large lists and it is not supported for lists that contain
reflected data types.  Users cannot use list-of-list as an argument because
of this limitation.

.. note::
   When passing a list into a JIT-compiled function, any modifications
   made to the list will not be visible to the Python interpreter until
   the function returns.  (A limitation of the reflection process.)

.. warning::
   List sorting currently uses a quicksort algorithm, which has different
   performance characterics than the algorithm used by Python.

.. _feature-list-initial-value:

Initial Values
''''''''''''''
.. warning::
  This is an experimental feature!

Lists that:

* Are constructed using the square braces syntax
* Have values of a literal type

will have their initial value stored in the ``.initial_value`` property on the
type so as to permit inspection of these values at compile time. If required,
to force value based dispatch the :ref:`literally <developer-literally>`
function will accept such a list.

Example:

.. literalinclude:: ../../../numba/tests/doc_examples/test_literal_container_usage.py
   :language: python
   :caption: from ``test_ex_initial_value_list_compile_time_consts`` of ``numba/tests/doc_examples/test_literal_container_usage.py``
   :start-after: magictoken.test_ex_initial_value_list_compile_time_consts.begin
   :end-before: magictoken.test_ex_initial_value_list_compile_time_consts.end
   :dedent: 12
   :linenos:

.. _feature-typed-list:

Typed List
''''''''''

.. note::
  ``numba.typed.List`` is an experimental feature, if you encounter any bugs in
  functionality or suffer from unexpectedly bad performance, please report
  this, ideally by opening an issue on the Numba issue tracker.

As of version 0.45.0 a new implementation of the list data type is available,
the so-called *typed-list*. This is compiled library backed, type-homogeneous
list data type that is an improvement over the *reflected-list* mentioned
above.  Additionally, lists can now be arbitrarily nested. Since the
implementation is considered experimental, you will need to import it
explicitly from the `numba.typed` module::

    In [1]: from numba.typed import List

    In [2]: from numba import njit

    In [3]: @njit
    ...: def foo(l):
    ...:     l.append(23)
    ...:     return l
    ...:

    In [4]: mylist = List()

    In [5]: mylist.append(1)

    In [6]: foo(mylist)
    Out[6]: ListType[int64]([1, 23])


.. note::
    As the typed-list stabilizes it will fully replace the reflected-list and the
    constructors `[]` and `list()` will create a typed-list instead of a
    reflected one.


Here's an example using ``List()`` to create ``numba.typed.List`` inside a
jit-compiled function and letting the compiler infer the item type:

.. literalinclude:: ../../../numba/tests/doc_examples/test_typed_list_usage.py
   :language: python
   :caption: from ``ex_inferred_list_jit`` of ``numba/tests/doc_examples/test_typed_list_usage.py``
   :start-after: magictoken.ex_inferred_list_jit.begin
   :end-before: magictoken.ex_inferred_list_jit.end
   :dedent: 12
   :linenos:

Here's an example of using ``List()`` to create a ``numba.typed.List`` outside of
a jit-compiled function and then using it as an argument to a jit-compiled
function:

.. literalinclude:: ../../../numba/tests/doc_examples/test_typed_list_usage.py
   :language: python
   :caption: from ``ex_inferred_list`` of ``numba/tests/doc_examples/test_typed_list_usage.py``
   :start-after: magictoken.ex_inferred_list.begin
   :end-before: magictoken.ex_inferred_list.end
   :dedent: 12
   :linenos:

Finally, here's an example of using a nested `List()`:

.. literalinclude:: ../../../numba/tests/doc_examples/test_typed_list_usage.py
   :language: python
   :caption: from ``ex_nested_list`` of ``numba/tests/doc_examples/test_typed_list_usage.py``
   :start-after: magictoken.ex_nested_list.begin
   :end-before: magictoken.ex_nested_list.end
   :dedent: 12
   :linenos:

.. _feature-literal-list:

Literal List
''''''''''''

.. warning::
  This is an experimental feature!

Numba supports the use of literal lists containing any values, for example::

  l = ['a', 1, 2j, np.zeros(5,)]

the predominant use of these lists is for use as a configuration object.
The lists appear as a ``LiteralList`` type which inherits from ``Literal``, as a
result the literal values of the list items are available at compile time.
For example:

.. literalinclude:: ../../../numba/tests/doc_examples/test_literal_container_usage.py
   :language: python
   :caption: from ``test_ex_literal_list`` of ``numba/tests/doc_examples/test_literal_container_usage.py``
   :start-after: magictoken.test_ex_literal_list.begin
   :end-before: magictoken.test_ex_literal_list.end
   :dedent: 12
   :linenos:

Important things to note about these kinds of lists:

#. They are immutable, use of mutating methods e.g. ``.pop()`` will result in
   compilation failure. Read-only static access and read only methods are
   supported e.g. ``len()``.
#. Dynamic access of items is not possible, e.g. ``some_list[x]``, for a
   value ``x`` which is not a compile time constant. This is because it's
   impossible to statically determine the type of the item being accessed.
#. Inside the compiler, these lists are actually just tuples with some extra
   things added to make them look like they are lists.
#. They cannot be returned to the interpreter from a compiled function.

.. _pysupported-comprehension:

List comprehension
''''''''''''''''''

Numba supports list comprehension.  For example::


  In [1]: from numba import njit

  In [2]: @njit
    ...: def foo(x):
    ...:     return [[i for i in range(n)] for n in range(x)]
    ...:

  In [3]: foo(3)
  Out[3]: [[], [0], [0, 1]]


.. note::
  Prior to version 0.39.0, Numba did not support the creation of nested lists.


Numba also supports "array comprehension" that is a list comprehension
followed immediately by a call to :func:`numpy.array`. The following
is an example that produces a 2D Numpy array::

    from numba import jit
    import numpy as np

    @jit(nopython=True)
    def f(n):
      return np.array([ [ x * y for x in range(n) ] for y in range(n) ])

In this case, Numba is able to optimize the program to allocate and
initialize the result array directly without allocating intermediate
list objects.  Therefore, the nesting of list comprehension here is
not a problem since a multi-dimensional array is being created here
instead of a nested list.

Additionally, Numba supports parallel array comprehension when combined
with the :ref:`parallel_jit_option` option on CPUs.

set
---

All methods and operations on sets are supported in JIT-compiled functions.

Sets must be strictly homogeneous: Numba will reject any set containing
objects of different types, even if the types are compatible (for example,
``{1, 2.5}`` is rejected as it contains a :class:`int` and a :class:`float`).
The use of reference counted types, e.g. strings, in sets is unsupported.

.. note::
   When passing a set into a JIT-compiled function, any modifications
   made to the set will not be visible to the Python interpreter until
   the function returns.

.. _feature-typed-dict:

Typed Dict
----------

.. warning::
  ``numba.typed.Dict`` is an experimental feature.  The API may change
  in the future releases.

.. note::
  ``dict()`` was not supported in versions prior to 0.44.  Currently, calling
  ``dict()`` translates to calling ``numba.typed.Dict()``.

Numba supports the use of ``dict()``.  Such use is semantically equivalent to
``{}`` and ``numba.typed.Dict()``. It will create an instance of
``numba.typed.Dict`` where the key-value types will be later inferred by usage.
Numba also supports, explicitly, the ``dict(iterable)`` constructor.

Numba does not fully support the Python ``dict`` because it is an untyped
container that can have any Python types as members. To generate efficient
machine code, Numba needs the keys and the values of the dictionary to have
fixed types, declared in advance. To achieve this, Numba has a typed dictionary,
``numba.typed.Dict``, for which the type-inference mechanism must be able to
infer the key-value types by use, or the user must explicitly declare the
key-value type using the ``Dict.empty()`` constructor method.
This typed dictionary has the same API as the Python ``dict``,  it implements
the ``collections.MutableMapping`` interface and is usable in both interpreted
Python code and JIT-compiled Numba functions.
Because the typed dictionary stores keys and values in Numba's native,
unboxed data layout, passing a Numba dictionary into nopython mode has very low
overhead. However, this means that using a typed dictionary from the Python
interpreter is slower than a regular dictionary because Numba has to box and
unbox key and value objects when getting or setting items.

An important difference of the typed dictionary in comparison to Python's
``dict`` is that **implicit casting** occurs when a key or value is stored.
As a result the *setitem* operation may fail should the type-casting fail.

It should be noted that the Numba typed dictionary is implemented using the same
algorithm as the CPython 3.7 dictionary. As a consequence, the typed dictionary
is ordered and has the same collision resolution as the CPython implementation.

Further to the above in relation to type specification, there are limitations
placed on the types that can be used as keys and/or values in the typed
dictionary, most notably the Numba ``Set`` and ``List`` types are currently
unsupported. Acceptable key/value types include but are not limited to: unicode
strings, arrays (value only), scalars, tuples. It is expected that these
limitations will be relaxed as Numba continues to improve.

Here's an example of using ``dict()`` and ``{}`` to create ``numba.typed.Dict``
instances and letting the compiler infer the key-value types:

.. literalinclude:: ../../../numba/tests/doc_examples/test_typed_dict_usage.py
   :language: python
   :caption: from ``test_ex_inferred_dict_njit`` of ``numba/tests/doc_examples/test_typed_dict_usage.py``
   :start-after: magictoken.ex_inferred_dict_njit.begin
   :end-before: magictoken.ex_inferred_dict_njit.end
   :dedent: 12
   :linenos:

Here's an example of creating a ``numba.typed.Dict`` instance from interpreted
code and using the dictionary in jit code:

.. literalinclude:: ../../../numba/tests/doc_examples/test_typed_dict_usage.py
   :language: python
   :caption: from ``test_ex_typed_dict_from_cpython`` of ``numba/tests/doc_examples/test_typed_dict_usage.py``
   :start-after: magictoken.ex_typed_dict_from_cpython.begin
   :end-before: magictoken.ex_typed_dict_from_cpython.end
   :dedent: 12
   :linenos:

Here's an example of creating a ``numba.typed.Dict`` instance from jit code and
using the dictionary in interpreted code:

.. literalinclude:: ../../../numba/tests/doc_examples/test_typed_dict_usage.py
   :language: python
   :caption: from ``test_ex_typed_dict_njit`` of ``numba/tests/doc_examples/test_typed_dict_usage.py``
   :start-after: magictoken.ex_typed_dict_njit.begin
   :end-before: magictoken.ex_typed_dict_njit.end
   :dedent: 12
   :linenos:

It should be noted that ``numba.typed.Dict`` is not thread-safe.
Specifically, functions which modify a dictionary from multiple
threads will potentially corrupt memory, causing a
range of possible failures. However, the dictionary can be safely read from
multiple threads as long as the contents of the dictionary do not
change during the parallel access.

Dictionary comprehension
''''''''''''''''''''''''

Numba supports dictionary comprehension under the assumption that a
``numba.typed.Dict`` instance can be created from the comprehension.  For
example::

  In [1]: from numba import njit

  In [2]: @njit
     ...: def foo(n):
     ...:     return {i: i**2 for i in range(n)}
     ...:

  In [3]: foo(3)
  Out[3]: DictType[int64,int64]<iv=None>({0: 0, 1: 1, 2: 4})

.. _feature-dict-initial-value:

Initial Values
''''''''''''''
.. warning::
  This is an experimental feature!

Typed dictionaries that:

* Are constructed using the curly braces syntax
* Have literal string keys
* Have values of a literal type

will have their initial value stored in the ``.initial_value`` property on the
type so as to permit inspection of these values at compile time. If required,
to force value based dispatch the :ref:`literally <developer-literally>`
function will accept a typed dictionary.

Example:

.. literalinclude:: ../../../numba/tests/doc_examples/test_literal_container_usage.py
   :language: python
   :caption: from ``test_ex_initial_value_dict_compile_time_consts`` of ``numba/tests/doc_examples/test_literal_container_usage.py``
   :start-after: magictoken.test_ex_initial_value_dict_compile_time_consts.begin
   :end-before: magictoken.test_ex_initial_value_dict_compile_time_consts.end
   :dedent: 12
   :linenos:

.. _feature-literal-str-key-dict:

Heterogeneous Literal String Key Dictionary
-------------------------------------------

.. warning::
  This is an experimental feature!

Numba supports the use of statically declared string key to any value
dictionaries, for example::

  d = {'a': 1, 'b': 'data', 'c': 2j}

the predominant use of these dictionaries is to orchestrate advanced compilation
dispatch or as a container for use as a configuration object. The dictionaries
appear as a ``LiteralStrKeyDict`` type which inherits from ``Literal``, as a
result the literal values of the keys and the types of the items are available
at compile time. For example:

.. literalinclude:: ../../../numba/tests/doc_examples/test_literal_container_usage.py
   :language: python
   :caption: from ``test_ex_literal_dict_compile_time_consts`` of ``numba/tests/doc_examples/test_literal_container_usage.py``
   :start-after: magictoken.test_ex_literal_dict_compile_time_consts.begin
   :end-before: magictoken.test_ex_literal_dict_compile_time_consts.end
   :dedent: 12
   :linenos:

Important things to note about these kinds of dictionaries:

#. They are immutable, use of mutating methods e.g. ``.pop()`` will result in
   compilation failure. Read-only static access and read only methods are
   supported e.g. ``len()``.
#. Dynamic access of items is not possible, e.g. ``some_dictionary[x]``, for a
   value ``x`` which is not a compile time constant. This is because it's
   impossible statically determine the type of the item being accessed.
#. Inside the compiler, these dictionaries are actually just named tuples with
   some extra things added to make them look like they are dictionaries.
#. They cannot be returned to the interpreter from a compiled function.
#. The ``.keys()``, ``.values()`` and ``.items()`` methods all functionally
   operate but return tuples opposed to iterables.

None
----

The None value is supported for identity testing (when using an
:class:`~numba.optional` type).


bytes, bytearray, memoryview
----------------------------

The :class:`bytearray` type and, on Python 3, the :class:`bytes` type
support indexing, iteration and retrieving the len().

The :class:`memoryview` type supports indexing, slicing, iteration,
retrieving the len(), and also the following attributes:

* :attr:`~memoryview.contiguous`
* :attr:`~memoryview.c_contiguous`
* :attr:`~memoryview.f_contiguous`
* :attr:`~memoryview.itemsize`
* :attr:`~memoryview.nbytes`
* :attr:`~memoryview.ndim`
* :attr:`~memoryview.readonly`
* :attr:`~memoryview.shape`
* :attr:`~memoryview.strides`


Built-in functions
==================

The following built-in functions are supported:

* :func:`abs`
* :class:`bool`
* :func:`chr`
* :class:`complex`
* :func:`divmod`
* :func:`enumerate`
* :func:`filter`
* :class:`float`
* :func:`getattr`: the attribute must be a string literal and the return type
  cannot be a function type (e.g. ``getattr(numpy, 'cos')`` is not supported as
  it returns a function type).
* :func:`hasattr`
* :func:`hash` (see :ref:`pysupported-hashing` below)
* :class:`int`: only the one-argument form
* :func:`iter`: only the one-argument form
* :func:`isinstance`
* :func:`len`
* :func:`min`
* :func:`map`
* :func:`max`
* :func:`next`: only the one-argument form
* :func:`ord`
* :func:`print`: only numbers and strings; no ``file`` or ``sep`` argument
* :class:`range`: The only permitted use of range is as a callable function
  (cannot pass range as an argument to a jitted function or return a range from
  a jitted function).
* :func:`repr`
* :func:`round`
* :func:`sorted`: the ``key`` argument is not supported
* :func:`sum`
* :func:`str`
* :func:`type`: only the one-argument form, and only on some types
  (e.g. numbers and named tuples)
* :func:`zip`

.. _pysupported-hashing:

Hashing
-------

The :func:`hash` built-in is supported and produces hash values for all
supported hashable types with the following Python version specific behavior:

Under Python 3, hash values computed by Numba will exactly match those computed
in CPython under the condition that the :attr:`sys.hash_info.algorithm` is
``siphash24`` (default).

The ``PYTHONHASHSEED`` environment variable influences the hashing behavior in
precisely the manner described in the CPython documentation.


Standard library modules
========================

``array``
---------

Limited support for the :class:`array.array` type is provided through
the buffer protocol.  Indexing, iteration and taking the len() is supported.
All type codes are supported except for ``"u"``.

``cmath``
---------

The following functions from the :mod:`cmath` module are supported:

* :func:`cmath.acos`
* :func:`cmath.acosh`
* :func:`cmath.asin`
* :func:`cmath.asinh`
* :func:`cmath.atan`
* :func:`cmath.atanh`
* :func:`cmath.cos`
* :func:`cmath.cosh`
* :func:`cmath.exp`
* :func:`cmath.isfinite`
* :func:`cmath.isinf`
* :func:`cmath.isnan`
* :func:`cmath.log`
* :func:`cmath.log10`
* :func:`cmath.phase`
* :func:`cmath.polar`
* :func:`cmath.rect`
* :func:`cmath.sin`
* :func:`cmath.sinh`
* :func:`cmath.sqrt`
* :func:`cmath.tan`
* :func:`cmath.tanh`

``collections``
---------------

Named tuple classes, as returned by :func:`collections.namedtuple`, are
supported in the same way regular tuples are supported.  Attribute access
and named parameters in the constructor are also supported.

Creating a named tuple class inside Numba code is *not* supported; the class
must be created at the global level.

.. _ctypes-support:

``ctypes``
----------

Numba is able to call ctypes-declared functions with the following argument
and return types:

* :class:`ctypes.c_int8`
* :class:`ctypes.c_int16`
* :class:`ctypes.c_int32`
* :class:`ctypes.c_int64`
* :class:`ctypes.c_uint8`
* :class:`ctypes.c_uint16`
* :class:`ctypes.c_uint32`
* :class:`ctypes.c_uint64`
* :class:`ctypes.c_float`
* :class:`ctypes.c_double`
* :class:`ctypes.c_void_p`

``enum``
--------

Both :class:`enum.Enum` and :class:`enum.IntEnum` subclasses are supported.

``math``
--------

The following functions from the :mod:`math` module are supported:

* :func:`math.acos`
* :func:`math.acosh`
* :func:`math.asin`
* :func:`math.asinh`
* :func:`math.atan`
* :func:`math.atan2`
* :func:`math.atanh`
* :func:`math.ceil`
* :func:`math.copysign`
* :func:`math.cos`
* :func:`math.cosh`
* :func:`math.degrees`
* :func:`math.erf`
* :func:`math.erfc`
* :func:`math.exp`
* :func:`math.expm1`
* :func:`math.fabs`
* :func:`math.floor`
* :func:`math.frexp`
* :func:`math.gamma`
* :func:`math.gcd`
* :func:`math.hypot`
* :func:`math.isfinite`
* :func:`math.isinf`
* :func:`math.isnan`
* :func:`math.ldexp`
* :func:`math.lgamma`
* :func:`math.log`
* :func:`math.log10`
* :func:`math.log1p`
* :func:`math.pow`
* :func:`math.radians`
* :func:`math.sin`
* :func:`math.sinh`
* :func:`math.sqrt`
* :func:`math.tan`
* :func:`math.tanh`
* :func:`math.trunc`

``operator``
------------

The following functions from the :mod:`operator` module are supported:

* :func:`operator.add`
* :func:`operator.and_`
* :func:`operator.eq`
* :func:`operator.floordiv`
* :func:`operator.ge`
* :func:`operator.gt`
* :func:`operator.iadd`
* :func:`operator.iand`
* :func:`operator.ifloordiv`
* :func:`operator.ilshift`
* :func:`operator.imatmul` (Python 3.5 and above)
* :func:`operator.imod`
* :func:`operator.imul`
* :func:`operator.invert`
* :func:`operator.ior`
* :func:`operator.ipow`
* :func:`operator.irshift`
* :func:`operator.isub`
* :func:`operator.itruediv`
* :func:`operator.ixor`
* :func:`operator.le`
* :func:`operator.lshift`
* :func:`operator.lt`
* :func:`operator.matmul` (Python 3.5 and above)
* :func:`operator.mod`
* :func:`operator.mul`
* :func:`operator.ne`
* :func:`operator.neg`
* :func:`operator.not_`
* :func:`operator.or_`
* :func:`operator.pos`
* :func:`operator.pow`
* :func:`operator.rshift`
* :func:`operator.sub`
* :func:`operator.truediv`
* :func:`operator.xor`

``functools``
-------------

The :func:`functools.reduce` function is supported but the `initializer`
argument is required.

.. _pysupported-random:

``random``
----------

Numba supports top-level functions from the :mod:`random` module, but does
not allow you to create individual Random instances.  A Mersenne-Twister
generator is used, with a dedicated internal state.  It is initialized at
startup with entropy drawn from the operating system.

* :func:`random.betavariate`
* :func:`random.expovariate`
* :func:`random.gammavariate`
* :func:`random.gauss`
* :func:`random.getrandbits`: number of bits must not be greater than 64
* :func:`random.lognormvariate`
* :func:`random.normalvariate`
* :func:`random.paretovariate`
* :func:`random.randint`
* :func:`random.random`
* :func:`random.randrange`
* :func:`random.seed`: with an integer argument only
* :func:`random.shuffle`: the sequence argument must be a one-dimension
  Numpy array or buffer-providing object (such as a :class:`bytearray`
  or :class:`array.array`); the second (optional) argument is not supported
* :func:`random.uniform`
* :func:`random.triangular`
* :func:`random.vonmisesvariate`
* :func:`random.weibullvariate`

.. warning::
   Calling :func:`random.seed` from non-Numba code (or from :term:`object mode`
   code) will seed the Python random generator, not the Numba random generator.
   To seed the Numba random generator, see the example below.

.. code-block:: python

  from numba import njit
  import random

  @njit
  def seed(a):
      random.seed(a)

  @njit
  def rand():
      return random.random()


  # Incorrect seeding
  random.seed(1234)
  print(rand())

  random.seed(1234)
  print(rand())

  # Correct seeding
  seed(1234)
  print(rand())

  seed(1234)
  print(rand())


.. note::
   Since version 0.28.0, the generator is thread-safe and fork-safe.  Each
   thread and each process will produce independent streams of random numbers.

.. seealso::
   Numba also supports most additional distributions from the :ref:`Numpy
   random module <numpy-random>`.

``heapq``
---------

The following functions from the :mod:`heapq` module are supported:

* :func:`heapq.heapify`
* :func:`heapq.heappop`
* :func:`heapq.heappush`
* :func:`heapq.heappushpop`
* :func:`heapq.heapreplace`
* :func:`heapq.nlargest` : first two arguments only
* :func:`heapq.nsmallest` : first two arguments only

Note: the heap must be seeded with at least one value to allow its type to be
inferred; heap items are assumed to be homogeneous in type.


Third-party modules
===================

.. I put this here as there's only one module (apart from Numpy), otherwise
   it should be a separate page.

.. _cffi-support:

``cffi``
--------

Similarly to ctypes, Numba is able to call into `cffi`_-declared external
functions, using the following C types and any derived pointer types:

* :c:expr:`char`
* :c:expr:`short`
* :c:expr:`int`
* :c:expr:`long`
* :c:expr:`long long`
* :c:expr:`unsigned char`
* :c:expr:`unsigned short`
* :c:expr:`unsigned int`
* :c:expr:`unsigned long`
* :c:expr:`unsigned long long`
* :c:expr:`int8_t`
* :c:expr:`uint8_t`
* :c:expr:`int16_t`
* :c:expr:`uint16_t`
* :c:expr:`int32_t`
* :c:expr:`uint32_t`
* :c:expr:`int64_t`
* :c:expr:`uint64_t`
* :c:expr:`float`
* :c:expr:`double`
* :c:expr:`ssize_t`
* :c:expr:`size_t`
* :c:expr:`void`

The ``from_buffer()`` method of ``cffi.FFI`` and ``CompiledFFI`` objects is
supported for passing Numpy arrays and other buffer-like objects.  Only
*contiguous* arguments are accepted.  The argument to ``from_buffer()``
is converted to a raw pointer of the appropriate C type (for example a
``double *`` for a ``float64`` array).

Additional type mappings for the conversion from a buffer to the appropriate C
type may be registered with Numba. This may include struct types, though it is
only permitted to call functions that accept pointers to structs - passing a
struct by value is unsupported. For registering a mapping, use:

.. function:: numba.core.typing.cffi_utils.register_type(cffi_type, numba_type)

Out-of-line cffi modules must be registered with Numba prior to the use of any
of their functions from within Numba-compiled functions:

.. function:: numba.core.typing.cffi_utils.register_module(mod)

   Register the cffi out-of-line module ``mod`` with Numba.

Inline cffi modules require no registration.

.. _cffi: https://cffi.readthedocs.org/