Skip to content

Blog

Python Fundamental

  • Fundamental of Python, a Jump start
  • Python Cheat Sheet for Leetcode - LeetCode Discuss
  • Create a list with size:
    • python
      [None] * 10
      [] * 10 #THIS WONT WORK
      list(range(10))  # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
      [{}] * 3
      [[]] * 3
      
  • Sort/Heap Sorting HOW TO
    • Note: sorted() will return a sorted list, it not change the original one while list.sort() will change the original list
  • Key Func:
    • a func to be called on each list elem, e.g. key=str.lower
    • it can be a lambda, e.g. key = lambda student: student.first_ name
    • The Key Functions are used in {{embed(((64a3827c-147d-443e-bb06-6b0e9f3ef2e0)))}}
  • operator **from** **operator** **import** itemgetter, attrgetter
    • 2nd and then 3rd element in tuple/list: sorted(student_{ tuples,} key=itemgetter(2, 3))
    • sort based on class attribute: sorted(student_{ objects,} key=attrgetter('grade', 'age'))
    • Note: itemgetter for list/tuple and attrgetter is for class
  • Comparison Functions How does the functools cmp_ to_ key function works in Python? - GeeksforGeeks
  • functools.cmp_ to_ key(callable)
    • A comparison/callable function is any callable that accepts two arguments, compares them, and returns a negative number for less-than, zero for equality,
    • example:
    • python
      import functools
      def mycmp(student1, student2):
        print("comparing ", student1, " and ", student2)
        if student1.age > student2.age:
            return 1
        elif student1.name < student2.name:
            return -1
        else:
            return 0
      print(sorted([student("James", 12), student("Mike", 11)], key=functools.cmp_to_key(mycmp)))
      
  • Iterator term:: An iterator is a Python object that implements a specific interface. iter___ return instance of iterator and next() method steps the iterator on cycle and return a value to next object id:: 65137d16-a06a-40e4-b28e-5fa4474017d5
  • functools: cache
    • @functools.cache(user_ function)
    • python
      import functools
      @functools.cache
      def factorial(n):
          return n * factorial(n-1) if n else 1
      factorial(10)      # no previously cached result, makes 11 recursive calls
      
  • list comprehension
    • When to Use a List Comprehension in Python – Real Python
    • baisic
      • python
        List = [character for character in [1, 2, 3]]
        print(List) #[1, 2, 3]
        [i for i in range(11) if i % 2 == 0] #[0, 2, 4, 6, 8, 10]
        matrix = [[j for j in range(3)] for i in range(3)] # 3x3 matrix
        
        lis = [num for num in range(100)
               if num % 8 == 0 if num % 10 == 0] # [0, 40, 80]
        
        string = 'Geeks4Geeks'
        # Toggle case of each character
        List = list(map(lambda i: chr(ord(i) ^ 32), string))
        # ['g', 'E', 'E', 'K', 'S', '\x14', 'g', 'E', 'E', 'K', 'S']
        
        # Reverse each string in tuple
        List = [string[::-1] for string in ('Geeks', 'for', 'Geeks')]
        # ['skeeG', 'rof', 'skeeG']
        
        # Remove Multiple Values
        ls = ['One', 'Two', 'Three', 'Three']
        to_remove = ['Three', 'Four']
        ls = [x for x in list(ls) if x not in to_remove]
        #Tree BFS
        [child for child in (n.left, n.right) if child]
        
  • Map, Filter, Reduce and Zip id:: 6506c411-3211-4b0c-9647-637d211344b3
    • Map, Filter, Reduce - Learn Python - Free Interactive Python Tutorial
      • Map and Filter return iterator (not list) Map
      • Map(func, *iterables)
        • Samples
          list(map(str.upper,  ['alfred', 'tabitha', 'william', 'arla']))
          list(map(round,  [3.56773, 5.57668, 4.00914, 56.24241, 9.01344, 32.00013], range(1, 7)))
          # 3rd parameter is parameter to round
          list(map(lambda x, y: (x, y), ['a', 'b', 'c', 'd', 'e'], [1, 2, 3, 4, 5]))
          list(map(lambda x, y, z: (x+y)*z, [1, 2, 3, 4, 5], [10, 20, 30, 40, 50], [1, 2, 3, 4, 5]))
          #[11, 44, 99, 176, 275]
          
      • Filter(func, iterable)

        • list(filter(lambda v: v > 75, [66, 90, 68, 59, 76, 60, 88, 74, 81, 65]))
        • filter and list comprehension

        [<exp1> for <var> in <iterable> if <logic expression>]
        [x for x in l if x%2]
        
        - reduce(func, iterable[, initial]) - python
        from functools import reduce
        numbers = [3, 4, 6, 9, 34, 12]
        def custom_sum(first, second):
            return first + second
        result = reduce(custom_sum, numbers)
        print(result)
        
        - Another using reduce to implement #Trie [[DSA/Trie]] - python
        Trie = lambda: defaultdict(Trie) #constructor return default dict
        trie = Trie() # dict to lambda
        END = True
        
        # Insert words into the trie
        words = ["apple", "banana", "apricot", "bear", "beach"]
        for word in words:
            reduce(dict.__getitem__, word, trie)[END] = word
        
        - e.g. word ‘apple’. then trie[‘a’] dict{‘p’: dict{‘p’: dict{‘l’: dict{‘e’: dict{True:’apple’}}}}} - Zip - creates an iterator that will aggregate elements from zero to more iterables. - zip([1, 2]); zip([1, 2], [‘a’, ‘b’]) - zip most useful to create <> - List comprehensions id:: 650c181a-1ea2-475c-a09b-35a4aa6ecd39 [x * y for x, y in zip([1, 2, 3], [3, 4, 5])] - - Heap #heapq - Operations: - peek: there are no peek, use heap[0] instead - heappush - heappop - python
        h = []
        for value in [1, 3, 5, 7, 9, 2, 4, 6, 8, 0]:
            heappush(h, value)
        return [heappop(h) for i in range(len(h))]
        
        - heappushpop(heap, item) push item and pop and return top element - heapify(x) : transform list x to heap, inplace O(n) - heapreplace(heap, item) pop smallest item and push new item, raise error if empty. It is efficient for fixed size heap. It works like poppush - merge: merge multiple sorted input into a single sorted. It return a iterable(not al ist) - nlargest & nsmallest: return list of n largest/smallest elements, Equivalent to: sorted(iterable, key=key, reverse=True/False)[:n] - Counter - Python’s Counter: The Pythonic Way to Count Objects – Real Python - counter.update(): the implementation provided by Counter adds existing counts together. It also creates new key-count pairs when necessary. - python
        >>> from collections import Counter
        >>> letters = Counter({"i": 4, "s": 4, "p": 2, "m": 1})
        >>> letters.update("missouri")
        >>> letters
        Counter({'i': 6, 's': 6, 'p': 2, 'm': 2, 'o': 1, 'u': 1, 'r': 1})
        >>> sales = Counter(apple=25, orange=15, banana=12)
        >>> # Use a counter
        >>> monday_sales = Counter(apple=10, orange=8, banana=3)
        >>> sales.update(monday_sales)
        >>> sales
        Counter({'apple': 35, 'orange': 23, 'banana': 15})
        
        - keys(): list of all keys - values(): list of all values - python
        cnt = Counter("AABC")
        total = sum(cnt.values())   # 4
        
        - most_ common() This method returns a list of (object, count) sorted by the objects’ current count - - Enum - python
        from enum import Enum
        
        # class syntax
        class Color(Enum):
            RED = 1
            GREEN = 2
            BLUE = 3
        
        # functional syntax  👍
        Color = Enum('Color', ['RED', 'GREEN', 'BLUE'])
        my_color = Color.RED
        
        - Const/Final - python
        from typing import Final
        
        PI: Final =  3.14
        
        - [[Comments]] - [[Sep 21st, 2023]] - ((650c181a-1ea2-475c-a09b-35a4aa6ecd39)) -

Python OOP and MetaProgramming

  • Python+Deep+Dive+4.pdf
  • Properties
    • In many languages direct access to attributes is highly discouraged Instead the convention is to make the attribute private, and create public getter and setter methods
    • Python property
      • ((65211fdc-5010-4ade-975c-c0cf9b480269))
      • ((65212021-413d-4bcd-a1c1-81a964e79a45))
    • property decorator @property
      • ((652120be-a7ed-40a4-9429-a7b7c1d4eb95))
    • ReadOnly Properties
      • ((6521213b-5183-4825-bca2-9dec4e4565c2))
  • Class Scope
  • Enumerators And Alias

    • Enum class

    class Color(Enum):
      red = 1
      crimson = 1
      carmine = 1
      blue = 2
      black = 3
    
    - auto values - enum.auto() generate a auto values for enum - MetaProgramming - type - type is a class

      ``` python
      class type:
        def __init__(self):
            在空值初识话数据
    
        def __new__(self):
              # __new__ from object
            创建->创建类
      ```
        - create a class with `type`: `type(class_name, class_base, class_dict)`
    - `type` allow you create new class programmly
    
    • Metaclass

      • The class used to create a class, is called metaclass of that class, e.g. MyType is metaclass of Person
      • By dflt, type is used to create a new class, but now if metaclass specified, it will be replace type
      • Create a new class from MyType MyType

      class MyType(type):
      
          def __new__(cls, *args, **kwargs):
              xx = super().__new__(cls, *args, **kwargs)
              return xx
      
      * method 1 use MyType()

      Foo = MyType("Foo", (object,), {"v1": 123, "func": lambda self: 999})
      
      * method 2 use metaclass

      class Foo(object, metaclass=MyType):
          v1 = 123
          def func(self):
              return 999
      
      - If Base class created with metaclass, all child/grandchild class of Base will be create with same metaclass - ((652124f8-b924-4054-8d9b-e56256794d28)) -

Python 3.10

Structural Pattern Matching, python version of switch

def respond(language):
    match language:
        case "Java" | "Javascript":  # multiple pattern match
            return "Love those braces!"
        case "Python":
            return "I'm a lumberjack and I don't need no braces"
        case _:  #default
            return "I have no clue!"

We could match against one or more literals by using the OR pattern |

Capturing match

def op(command):
    match command:
        case ["move", ("F" | "B" | "L" |"R") as direction]:
            return symbols[direction]
        case "pick":
            return symbols["pick"]
        case "drop":
            return symvols["drop"]
        case _:
            raise ValueError(f"{command} does not compute!")

It match "move F", "move B" "move L" "move R"

*union* ~|~

s1 = {'a', 'b', 'c'}
s2 = {'c', 'd'}

s1 | s2 #[[{'a', 'b', 'c', 'd'}]]

d1 = {'c': 3, 'a': 1, 'b': 2}
d2 = {'d': 40, 'c': 30}
d1 | d2  # {'c': 3, 'd': 40, 'a': 1, 'b': 2}

assignment expressions :=

it assign value and return the values

(x:=1+3) # output 4 and x=4
a = (x := 10 + 20)  # same as a=x=30

examples

even_results = [
    result
    for i in range(10)
    if (result := slow_function(i, i)) % 2 == 0
]
random.seed(0)
def even_random(n):
    cnt = 0
    while (cnt := cnt + 1) <= n:
        if (number := random.randint(0, 10)) % 2 == 0:
            yield number

*

python basic

  • Python+Deep+Dive+1.pdf
  • Refresher
    • Multi-line Statement in Python:
      • To extend the statement to one or more lines we can use braces {}, parentheses (), square [], semi-colon “;”, and continuation character slash “\”.
      • For string you can use ''' or """ for a multiple line string
        list = [5,
                4, 3, 2, 1
                ]
        print('Initializing a list using the\
         Implicit multi-line statement', list)
        g = """geeks
        for
        geeks"""
        # Initializing a mathematical expression
        # using the Implicit multi-line statement.
        add = (50 +
               40 -
               52)
        if a \
          and b:
          pass
        
    • Variable Naming
      • ((6505ac52-5d0c-4272-8c97-ea3ab05e7dcd))
    • Condition Expression
      • 5 < a < 7 vs 5<a and a<7
      • ternary
        • b = 1 if a < 5 else 2
        • var = exp1 if con-exp2 else exp2
    • Continue, break, while, else, try, catch, finally

      • The continue statement skips the current iteration of a loop and continues with the next iteration.
      • finally will be executed even with continue|break
      • loop altogether, and the program continues after the loop. aka goto loop_exit
      • Inside try-except-finally

      # in a for Statement
      for x in range(2):
        try:
            print('trying...')
            continue   #break
            print('still trying...')
        except:
            print('Something went wrong.')
        finally:
            print('Done!')
      print('Loop ended.')
      # Prints trying...
      # Prints Done!
      # Prints trying...
      # Prints Done!
      # Prints Loop ended.
      
      - finally clause is executed before starting(break; exit) the next iteration. - break with for/while-else - If the loop terminates prematurely with break, the else clause won’t be executed.

        ``` python
        # Break the for loop at 'blue'
        colors = ['red', 'green', 'blue', 'yellow']
        for x in colors:
            if x == 'blue':
                break
            print(x)
        else:
            print('Done!') #will never executed
        # Prints red green
        ```
      
      • Variable
        • Everything is object
      • e.g. int is
        • Python Garbage Collection
      • Viewing reference counts in Python

      >>> import sys
      >>> a = 'my-string'
      >>> sys.getrefcount(a)
      >>> del a  # set ref count to 0
      >>> import gc
      >>> gc.get_count()
      (595, 2, 1)
      >>> gc.collect()
      577
      >>> gc.get_count()
      (18, 0, 0)
      
      - Disabling the garbage collector - Dismissing Python Garbage Collection at Instagram | by Instagram Engineering | Instagram Engineering - When share memory or object reused repeatedly disable GC can save - Python object id() is used to get location of a object - Type - Python is Dynamic typed, use type() to get object type - Python access variable data value through reference. Object can bind to different type (change reference) durning execution - Mutable vs immutable - ((650790fd-e50a-4807-9705-97324dbb967b)) - Inmutable: create a new object if value changed, id(my_var) changes when value changed. - Why it is important? - Side-effect - performance - Notes - mutable can change id. E.g. list.appen(val) vs list += [val] vs list = list+[val]. The last list=list+[val] creates a new object - use append or += when possible - A immutable object can have mutable objects

        ``` python
        t = ([1], [3])
        t[0].append(2)
        ```
      - immutable are safe from unintended side-effect (e.g. func call)
      - to prevent side-effect, use copy() to shallow copy a mutable object(e.g. list)
      - Share reference
      
      • python share reference by default ((65079bf5-d1fd-48ef-aa24-680ec28b97cd))
      • With mutable objects e.g. list, Python will NEVER create shared reference. This may confusing with

      a=[1, 2]
      b=a #reference
      c=[1, 2]  #does not share with a/[1,2]
      
      - Python pr-create values [-5, 256] so if value in that range, it will share reference - Equality - is identity operator id(a) == id(b) - Sample

      a = 10
      b = a
      a is b
      a == b
      a = [1]
      b = [1]
      a is not b
      
      - All objects of None are same. Same id and same value a is None - Numbers - Python does handle allication memory/bytes for numbers based on the value, e.g. \(2^{1000}\) use 160 bytes - The larger the number the more memory - floor is largest(in stardard number order)

      floor(3.3) -> 3
      floor(-3.3) -> -4
      
      - / divide -> float - // divide floor -> int - % reminder -> int - int(10.9) -> truncation 10 - Integer / int - constructor int(10.1); int(True); int(Decimal("10.2")); int("10") - With Base. int("1010", 2) ; int("1010", base=2) - bin() -> "0b1010"; oct() -> "0o12"; hex() -> "0xa" - sign(x): 1 if x >= 0 else -1 - Rational and Fraction - Franction -
      From factions import Fraction
      Fraction(3,4)  #3/4   numerartor, denominator
      Fraction(3.4)
      Fraction('3.4')
      Fraction('3.4') * Fraction(3.4)
      Fraction(math.sqrt(2)) # will be comn
      In [17]: y=Fraction(sqrt(2))
      In [18]: y
      Out[18]: Fraction(6369051672525773, 4503599627370496)
      y.limit_denominator(10)
      
      - use y.limit_{ denominator(10)} to round denominator to close to 10 - Float - IEEE 754 double-precision binary float aka binary 64 - sing 1bit - exponent 11bit - significant 52 bit - Equality - math.isclose(a, b, *, rel​​​​​_​​​​​​​tol=1e - 9, abs​​_​​tol=0.0) - e.g. math.isclose(3.3, 1+2.3000) - round(val, digits) e.g. round(3.1415, 2) -> 3.14 - You should always use isclose for compare float - Float to int - truncation|math.trunc(), floor, ceiling, rounding - trunc keep the int part. same as int(val) - floor largest integer LE to the val - ceiling min{i >= x} - round(val, n). closest multiple of 10^{ -n} , n default to 0 so round(val) -> int(val) - round(1.25, 1) -> 1.2 (to nearest val with even least significant digit) - Banker’s Rounding - decimal - Unlike floats, Python represents decimal numbers exactly. And the exactness carries over into arithmetic - Decimal always associates with a context that controls the following aspects: - Precision during an arithmetic operation (default 28) - Rounding algorithm - Sample

        ``` python
        import decimal
        from decimal import Decimal
        decimal.getcontext().prec = 2
        pi = Decimal('3.14159')
        print( pi * radius * radius )
        pi = Decimal(sign, (d1, d2, d3, ...), exp)
        pi = Decimal (0, (3, 1, 3, 1, 5), 4)
      
        10.0 == Decimal('10.0')  # .0 will be use
        0.1 != Decimal('0.1') #  print('{:.20f}'.format(0.1)) 0.10000000000000000555
        ```
      - Decimal arithmetic operators
          - `//` and `%`
          - `Decimal(a)//Decimal(b)` -> `trunc(a/b)`
      
      • Complex number
        • use cmath
        • Boolean
      • Boolean is subclass of int but it is used in totally different way. Every object in python has a truth value (truthiness)
      • bool class
        • True and False
        • isubclass of int issubclass(bool, int)
        • wired truth of bool
        • isinstance(True, int) -> True
        • True -> 1; False->0 (int(True)->1; True == 1 ; True > False)
          • True + True = 2
        • They are singleton object
        • You can use both a==True and a is True because it is singleton
      • Truthy
        • All objects are True except
          • None
          • False
          • classes implement __bool__ or __len__ that return False or 0. Default of __bool__ is return self != 0 e.g. bool(100) will execute int(100).__bool__() and therefore return result of 100 != 0
          • Based on above following is False:
            • 0 in any numeric type (0, 0.0, 0+0j)
            • empty collections (list, tuple, string, dict set)
        • for [[javascript]] ((650504dd-d2ad-48f6-b26b-b98d6c5e99dd))

      • Logical operation precedence
        • ()
        • compare >, < == !=
        • in is
        • and
        • or
      • Short-Circuit
        • Be care that when short-circuited, part of the expression may not executed
      • Logic operations

        • X or Y is equal to X if X else Y

        x = 32
        y = 7
        print(x or y)  # 32
        x = 0
        y = 'abc'
        print(x or y) # abc
        
        - X and Y X if not X else Y -
        x = 10
        y =  x and 20/x  # y = 2
        
        z = (s and s[0]) or ''  # if s: return s[0]; else return ''
        
        - Comparison operators - chained comparisons - In Python, chaining comparison operators is a way to simplify multiple comparison operations by stringing them together using logical operators. This is also known as “chained comparisons” or “chained comparison operators”. - a==b==c - a<b<c - a<b>c - a>b<c - a<b<c<d -
        exp1 = a <= b < c > d is not e is f
        
        - Function - Argument and parameter - Positional and keyword arguments - It following C++ rule. If a positional parameter is defined with default value, every positional parameter after it must given a default value

        def myfun(a, b=100, c=0): # a positional
          pass
        myfun(1)
        myfun(1, 2)
        myfun(1, 2, 3)
        
        - keyword argument -
        myfun(a=1, b=2, c=3)
        my(1, 2, c=3)
        myfun(1, c=2) #b skipped and will use default value of 100
        myfun(c=3, a=1, b=2) # it is ok not follow same order
        myfun(a=1, 2, 3) # ❌not correct, the rest after `a=1` must be named
        
        - Function arguments list is ((650994bb-61c3-4e01-b5a0-e7eeda531610)) - *arg to accept variable length arguments - example

          ``` python
          def func(a, b, c)
            pass
          l = [1, 2, 3]
          func(*l)  #unpack  l from list to numbers
        
          def func2(a, b, *args)
            print(a, b, args)
          func2(10, 20, 1, 2, 3)  # 10, 20 (1, 2, 3)
        
          def avg((args):
            return args and xxxxsum(args)/len(args)
          ```
        - all arguments after `*arg` must be ^^keyword^^ arugments
        
          ``` python
          def func2(a, b, *args, d)
            print(a, b, args)
          func(1, 2, 3, 4, d=5)
          ```
        - ^^*^^ without name `def func(*, d)` means there are no more positional args, you must provides keyword arguments
            - it means you can not pass arguments other than <<keyword argument>>
            - e.g `func(d=32)`
        - ^^/^^ : `def mod(x, y, /)` means x, and y are << position ONLY parameters >>
        
        • **kwargs to unpack dict

          • used to group keywords arguments
          • can be spicified even if positional arguments not been exausted
          • NO parameters can come after **kwargs
          • Sample

          def func(*, d, **kwargs):
            print(d, kwargs)
          func(d=1, a=2, b=3)   # d=1, kwargs = {'a': 2, 'b':3}
          def func2(*args, **args):
            print(args, kwargs)
          func(1, 2, a=10, b=20)  # (1, 2) {a:10, b:20}
          
          - you can not do this: func(a, b, **, ***kwargs) - Default arguments #pitfall - It created once, the value should be treat as const. If you need the argument to be variable, e.g. datetime.now() do not use default argument. - Solution: default to None - Another case, if initialize a default parameter with collection (e.g. list, dict). As same reason above, it will freeze a collection object to variable, if the function reused, the collection object will be resused and it may have incorrect values - Solution: default to Nono, not empty collection - sample:

            ``` python
            def func(l =[]):
              l.appen(1)
              return l
          
            print(func())
            print(func())  # 1, 1
            ```
          - This can also be used for recursion remember the results #memoization
              - factorial example:
          
                ``` python
                def factorial(n, cache={}):
                  if n<1:
                    return 1
                  elif n in cache:
                    return cache[n]
                  else:
                    k = factorial(n-1)*n
                    cache[n]=n*factorial(n-1)
                    return k
                ```
          - First-Class Functions
              - High order functions
                  - A function take parameter of another function as arguments
              - Docstring PEP 257
                  - 🧑‍🏫A **docstring** is a string literal that occurs as the ^^first statement^^ (exclude comments) in a module, function, class, or method definition. Such a docstring becomes the _{ doc _ special} attribute of that object.
                  - function docstr stored in function. _ doc _
                  - Sample:
          

        def root():
            """Return the pathname of the KOS root directory."""
            global _kos_root
            if _kos_root: return _kos_root
              ...
        def function(a, b):
            """function(a, b) -> list"""
        
        def complex(real=0.0, imag=0.0):
            """Form a complex number.
            Keyword arguments:
            real -- the real part (default 0.0)
            imag -- the imaginary part (default 0.0)
            """
        
        - Annotations PEP 3107 - 🧑‍🏫Function annotations are arbitrary python expressions that are associated with various part of functions. - Benefit: help string(e.g. with sphinx), compile check - it stored in __annotations__ in K:V format - <> - type hints is one form of annotations - Sample:

        def f(a:str = 'a', b: [1,2,3]) ->str:
          ...
        def f2(a:str) -> 'a repeated ' + str(max(x,y)) + ' times'
          ...
        
        - Lambda Expressions - Limitations - single line expression - no assignment - no annotations - Syntax

        lambda arguments : expression
        
        lambda s: s[::-1].upper()
        def appply_func(x, fn):
          return fn(x)
        apply_func(2, lambda x: x**)
        
        - Usages - use in sorted key

          ``` python
          sorted(d, key = lambda s : s.upper())
          # randomisze a string
          sorted(d, key = lambda x: random.random())
          ```
            - Introspection
        
        • __name__ function name
        • __defaults__, __kwdefaults__ defaults values
        • __code__ the code objects includes
          • co_varnames : parameters
          • co_argcount
        • <> module
          • isroutine: func or method
          • getsource | getmoudle | getcomments | signature
            • callable
        • a object like (but not limited to) <> and <>
        • callable() check if a object is callable
        • <> is callable
        • generators, coroutines, asynchronous generators
        • any objects implements __call__ <> - {{embed(((6506c411-3211-4b0c-9647-637d211344b3)))}} - Partial function
        • samples: - python from functools import partial # A normal function def f(a, b, c, x): return 1000*a + 100*b + 10*c + x # A partial function that calls f with # a as 3, b as 1 and c as 4. g = partial(f, 3, 1, 4) # Calling g() print(g(5)) # A partial function with b = 1 and c = 2 add_part = partial(add, c = 2, b = 1) # Calling partial function print(add_part(3))
        • Use
          • Callback signature
          • Integration with other API

            - Operator

        • The most used functions

        Method Signature Behaves like
        abs abs(a) abs(a)
        add add(a,b) a+b
        and_ and_(a,b) a&b
        concat concat(a,b) string: a+b
        contains contains(a,b) b in a
        countOf countOf(a,b) a.count(b)
        delitem delitem(a,b) del a[b]
        delslice delslice(a,b,c) del/a[b:c]
        div div(a,b) a/b
        eq eq(a,b) a==b
        floordiv floordiv(a,b) a//b
        ge ge(a,b) a>=b
        getitem getitem(a,b) a[b]
        getslice getslice(a,b,c) a[b:c]
        gt gt(a,b) a>b
        indexOf indexOf(a,b) a.index(b)
        invert, inv invert(a), inv(a) ~a
        is is(a,b) a is b
        is_not is_not(a,b) a is not b
        le le(a,b) a<=b
        lshift lshift(a,b) a<<b
        lt lt(a,b) a<b
        mod mod(a,b) a%b
        mul mul(a,b) a*b
        ne ne(a,b) a!=b
        neg neg(a) -a
        not_ not_(a) not a
        or_ or_(a,b) a
        pos pos(a) +a
        repeat repeat(a,b) a*b
        rshift rshift(a,b) a>>b
        setitem setitem(a,b,c) a[b]=c
        setslice setslice(a,b,c,d) a[b:c]=d
        sub sub(a,b) a-b
        truediv truediv(a,b) a/b # “true” div -> no truncation
        truth truth(a) not not a, bool(a)
        xor_ xor(a,b) a^{ b}
        - Most operator as dunder operater as well e..g. le __le__
        - dunder is used for object compare e.g. a<b , operator used for lambda and place need a function as argument
        - Samples

        reduce(mul, [1,2,3])  # 6
        
        - In-place operator - iadd iand iconcat ifloordiv ilshift imod imul imatual ior ipow isub ixor itruediv - attrgetter(attr) - 返回一个可从操作数中获取 attr 的可调用对象。 如果请求了一个以上的属性,则返回一个属性元组。 属性名称还可包含点号。 例如: - 在 f = attrgetter(‘name’) 之后,调用 f(b) 将返回 b.name。 - 在 f = attrgetter(‘name’, ‘date’) 之后,调用 f(b) 将返回 (b.name, b.date)。 - 在 f = attrgetter(‘name.first’, ‘name.last’) 之后,调用 f(b) 将返回 (b.name.first, b.name.last) - __getattr__ - If attr not existed in object this function will be called. e.g. obj.func_ont_exist or getattr(obj, 'not_existed') - __getattribute__ - always triggered when reference a attribute inside a object - itemgetter(item|*item) - 返回一个使用操作数的 [[https://docs.python.org/zh-cn/3/library/operator.html#operator.__getitem__][]] 方法从操作数中获取 /item/ 的可调用对象。 如果指定了多个条目,则返回一个查找值的元组。 - sample

          ``` python
          itemgetter(1)('ABCDEFG') #B
          itemgetter(1, 3)('ABCDEFG')  #('B', 'C')
          ```
        
        • methodcaller(name, /, **args, ***kwargs)

          • 返回一个在操作数上调用 /name/ 方法的可调用对象。 如果给出额外的参数和/或关键字参数,它们也将被传给该方法

          def methodcaller(name, /, *args, **kwargs):
              def caller(obj):
                  return getattr(obj, name)(*args, **kwargs)
              return caller
          class MyClass:
            def test(self, arg):
              print("test", arg)
          obj = MyClass()
          testfun = attrgetter('test')(obj)
          testfun("aaa")
          methodcaller('test', 'aaa')(obj)
          
          - Closure - Abstract - Questions, keywords and cues
          :questions-keywords-cues:
                    - What do I already know?
                    - Strengths and weaknesses?
                    - When to apply this theory?
                    - How valid are the research methods?
                    - How strong is the evidence?
                    - How logical is the argument?
                    - How does this fit in to other research in the field?
                    - What do I need to find out next?
          
          :END:
          
          - When to apply this? -


          - abstract & reflect
          

          org :main-idea-checkbox: - What is this aims? - What is the their research question? - What is the author arguing? - What is their answer to the question? - What points support their argument? - What are their main reasons? - What evidence have they used to support their argument? - What’s the significance of these facts? - What principle are they based on? - How can I apply them?  How do they fit in with what I already know? - What’s beyond them? - What're supporting details and explanations? :END: - Main points - 📖 Scope - Scopes and namespace - python variable scopes - It is important when a symbol is hide/blocked by another symbol - built-in scope - print, True, False etc are located in bult-in scope. If a symbol is not in module scope or current LEG, search in built-in scope. This is #LEGB - Local Scope - inside a function. So it also called function local scope - Enclosing Scope - nonlocal scope - nonlocal - The nonlocal keyword is used in nested functions to declare that a variable refers to a variable in the nearest enclosing scope that is not global. This means if you have a nested function and you want to modify a variable from the outer (enclosing) function, you’d use nonlocal. Sometime the nonlocal variable also been called free variable - free variable - Use the keyword nonlocal to declare that the variable is not local. - When nonlocal can be omitted - if it is read after write or read only, you can skip nonlocal keyword - nested function create an outer and inner scope, use nonlocal keyword to refer to the outer scope instead of create a new local variable

        def outer():
            string = "Favtutor" # Local Variable
            def inner():
                nonlocal string #declaring a non local variable
                string= "Python Favtutor Classes" # Overwriting value of a variable string
                print("inner function:", string)
            inner()
            print("outer function:", string)
        
        outer()
        
        - Global scope - A variable created in the main body of the Python code is a global variable and belongs to the global scope. aka module scope or file scope. It spans a single file only - Global Keyword - If you need to create a global variable, but are stuck in the local scope, you can use the global keyword. - The global keyword makes the variable global.

          ``` python
          def myfunc():
            global x  # create a global variable
            x = 300
          myfunc()
          print(x)
          ```
            - Global and local
        
        • The main difference is that Global is used to access and modify global variables from within a function, while nonlocal is used to access and modify variables from the nearest enclosing scope that is not global.
        • when python encounter a func definition at compile time. It scans for labels/var that have assigned to them anywhere in the function. If the label has not been specified as global it is a local
        • Var referenced but not assigned anywhere in the func will not be local and python at runtime look for them in enclosing scopes
        • Nonlocal declarations in a local scope do not require the variable to be pre-bound (it declared in outer scope), which is another fundamental distinction between them. These variables must already have been bound in the surrounding namespace(outer function) to avoid syntax errors.
        • While a nonlocal statement allows for the alteration of an enclosing scope variable in the local scope, a global statement allows for the modification of a global variable in the local scope. Nonlocal variables must already exist, although global variables can be declared with brand-new variables.
        • Sample

        a = 10
        def f3():
          global a
          a = 100 # this refer to the a at line 1
        def f4():
          print(a)  # this refer to a at next line and will throw a runtime error
          a = 100
        
        - del a symbol in current scope - e.g

        print = lambda n: print(2**n)
        print(3)
        del print
        
        - Cell object - Cell objects are used to implement variables referenced by multiple scopes. For each such variable, a cell object is created to store the value; the local variables of each stack frame that references the value contains a reference to the cells from outer scopes which also use that variable. When the value is accessed, the value contained in the cell is used instead of the cell object itself. This de-referencing of the cell object requires support from the generated byte-code; these are not automatically de-referenced when accessed. Cell objects are not likely to be useful elsewhere. - ((650f9ddb-a02a-45cf-9524-8c395f42f66e)) - Cell is important to understand free variables used in closure - 📖 Closure - When closure created, it put the nonlocal and global variable into a dict ((650f9d5c-c95d-41d3-8f60-6d6ada6c32f2)) and closure can be introspect with __closure__ - if there is no ((65100808-8929-4581-a57b-c1f9470e4eb6)) there is no closure - Use closure to remember state - ((650fb53c-bad9-4c9b-87c7-acc3a3d2a8b8)) - When counter() created a closure. It include a reference to counter/cell. So each time fn() called, count will change - Multiple instance of Closures - Each time create a new closure, a new scope/env/capture will also created.

        f1 = counter()
        f2 = counter()
        f1() # -> 1
        f1() # -> 2
        f2() # -> 1
        
        - Shared scope and share reference/cell - Following code

        adders = []
        for n in range(1, 4):
          adders.append(lambda x: x + n)
        adders[0](1)  -> 4
        adders[1](1)  -> 4
        adders[2](1)  -> 4
        
        # vs
        adders = []
        for n in range(1, 4):
          adders.append(lambda x, y=n: x + y)
        adders[0](1)  -> 2
        adders[1](1)  -> 3
        adders[2](1)  -> 4
        
        - The reason n is 3 for each adder closure is: - n is a share scope object in for loop - it does not recreated each time - each adder has a ((650f9d5c-c95d-41d3-8f60-6d6ada6c32f2)) which is reference to n - It is important to know closure scope value is stored in ((650f9d5c-c95d-41d3-8f60-6d6ada6c32f2)) and store reference, when the value reference pointing to changed, closure value will also change . - Python does not evaluate free vars n until adders[i] func is called. And all of then refer to same n(3) when it is called - Replace Class with closure - In many cases, the only reason we might have a single-method class is to store additional state for the use in method. - A request class

        import requests
        class SourceTemplate:
            def __init__(self, url):
                self.url = url
            def load(self, **kwargs):
                return requests.get(self.url.format_map(kwargs))
        github = SourceTemplate('https://api.github.com/repositories?since={since}')
        github.load(since=200).json()
        
        - A closure implementation

        def sourcetemplete(url):
            def load(**kwargs):
                return requests.get(url.format_map(kwargs))
            return load
        load = sourcetemplete('https://api.github.com/repositories?since={since}')
        load(since=200).json()
        
        - Recites
        :howto-recite:
                  Cover the notetaking column with a sheet of paper.  Then, looking at the questions or cue-words in the question and cue column only, say aloud, in your own words, the answers to the questions, facts, or ideas indicated by the cue-words.
        
        :END:
        
        -


        - Summary
            - main points
        -
        
        • Decorator
          • Abstract
            • The outer function is called the decorator, which takes the original function as an argument and returns a modified version of it.
            • So, in the most basic sense, a decorator is a callable that returns a callable.
            • ((65101822-3477-4dbd-bf27-c6d41cb38579))
          • Questions, keywords and cues ```org :questions-keywords-cues:
            • What do I already know?
            • Strengths and weaknesses?
            • When to apply this theory?
            • How valid are the research methods?
            • How strong is the evidence?
            • How logical is the argument?
            • How does this fit in to other research in the field?
            • What do I need to find out next? :END: ```
            • What is Decorator?


        - abstract & reflect
        

        org :main-idea-checkbox: - What is this aims? - What is the their research question? - What is the author arguing? - What is their answer to the question? - What points support their argument? - What are their main reasons? - What evidence have they used to support their argument? - What’s the significance of these facts? - What principle are they based on? - How can I apply them?  How do they fit in with what I already know? - What’s beyond them? - What're supporting details and explanations? :END: - Main points - 📖 Define a decorator - Using nested functions

      def counter(fn):
          count = 0
          def inner(*args, **kwargs):
              nonlocal count
              count += 1
              print('Function {0} was called {1} times'.format(fn.__name__, count))
              return fn(*args, **kwargs)
          return inner
      
      def add(a, b=0):
        "return sun of two integers"
        return a + b
      add = counter(add)
      add(1, 2) # Function add was called 1 times 3
      
      - Pythonic way

      @counter
      def mult(a: float, b: float=1, c: float=1) -> float:
        "mult return products of three number"
        return a * b * c
      
      - There is a minor problems that mult is no longer mult after wrapped by operator. The __doc__ and __name__ changed. That when @wraps is needed

      def counter(fn):
          count = 0
          @wraps(fn)
          def inner(*args, **kwargs):
              nonlocal count
              count += 1
              print("{0} was called {1} times".format(fn.__name__, count))
          return inner
      
      It is same as this:

      def counter(fn):
          count = 0
      
          def inner(*args, **kwargs):
              nonlocal count
              count += 1
              print("{0} was called {1} times".format(fn.__name__, count))
          inner.__name__ = fn.__name__
          inner.__doc__ = fn.__doc__
          return inner
      
      And also this:
      def counter(fn):
          count = 0
      
          def inner(*args, **kwargs):
              nonlocal count
              count += 1
              print("{0} was called {1} times".format(fn.__name__, count))
          inner = wrap(fn)(inner)
          return inner
      
      - - 📖 Decorator with parameters (Decorator factory) - Decorator with parameters are wrapper around existing decorators. Decorator returns closure but decorator with parameters returns decorator. - A timer decorator with parameters

      # name factory is decorator factory which accepts parameters
      def factory(number):
        # name timer is actual decorator
          def timer(fn):
              from time import perf_counter
      
            # name inner is closure
              def inner(*args, **kwargs):
                  total_time = 0
                  for i in range(number):
                      start_time = perf_counter()
                      to_execute = fn(*args, **kwargs)
                      end_time = perf_counter()
                      execution_time = end_time - start_time
                      total_time += execution_time
                  average_time = total_time/number
                  print('{0} took {1:.8f}s on an average to execute (tested for {2} times)'.format(fn.__name__, execution_time, number))
                  return to_execute
      
              return inner
          return timer
      
      @factory(50)
      def function_1():
          for i in range(1000000):
              pass
      
      @factory(5)
      def function_2():
          for i in range(10000000):
              pass
      
      function_1()
      function_2()
      
      - Recites
      :howto-recite:
                Cover the notetaking column with a sheet of paper.  Then, looking at the questions or cue-words in the question and cue column only, say aloud, in your own words, the answers to the questions, facts, or ideas indicated by the cue-words.
      
      :END:
      
      -


      - Summary
      

      - main points

      • Tuple
      • (1) is not a tuple, it is a int, a single int of 1
      • 1, is a tuple
      • 1, 2, 3 is a tuple
      • () is used just to make code looks nicer
      • create a empty tuple using tuple(), () will do as well
        • Pack and unpack
      • Similar to ((650504dd-7dea-4c1a-9a18-71aa0e1f2200))
      • a, b, c = 1, 2, 3; a, b, c = [1, 2, 3]; (a, b, c) = [1, 2, 3] they works the same way
      • behind the scenes: a, b, c is a tuple
      • a, b, c = 'XYZ' a='X', b='Y', c='Z'
      • for e in 12, 10, 'hello'
      • swap: a, b = b, a

        python d = {'1': 1, '2':2} a, b = d # a can be '1' or '2'
      • extended unpack with * and ** - python a, *b = [1, 2, 3] # a=1, b = [2, 3] a, *b = 'XYZ' #b=['Y', 'Z'] a, *b, c = 'abcd' # b=['b', 'c']

        • unpack can be used other way around

        l1=[1, 2]
        l2=[3, 4]
        l = [*l1, *l2]  # 1, 2, 3, 4
        l3 = 'XYZ'
        [*l1, *l3] # [1, 2, 'X', 'Y', 'Z']
        
        - unpack dict with ** - merge dict

          ``` python
          d1={1:1}
          d2={1:1, 2:2}
          d3={1:2: 3:3}
          d = {**d1, **d, **d3}  # duplicated value will be overwrote
          ```
        
        • nested unpacking

        a, *b, (c, d, e) = [1, 2, 3, 'XYZ'] # b = [2, 3], c = X ...
        
        - Ignore some of fields when unpack

        a, *_, c = (1, 2, 3, 4, 5)   # -> c = 5
        a, *ignore, c = (1, 2, 3, 4, 5)   # -> c = 5
        
        - Named Tuple - namedtuple is a class factory return a subclass of tuple. You need a class_{ name} and array of element names to create it. The name can also be a string of space or , separated words Syntax:

      TupleClass = namedtuple(class_name, ['array', 'elements', 'string', 'format'])
      TupleClass = namedtuple(class_name,  'string of names')
      # Sample
      Point = namedtuple('Point', ['x', 'y'])
      # Now Point is a class
      p1 = Point(1, 3)
      print(p1.x, p1.y)
      p1 = Point(x=1, y=3)
      
      - Name tuple is tuple so it can use all tuple operations

      x = Point[0]
      x = Point[:1]
      
      - Extend a tuple

      Point1D = namedtuple('Point1D', ['x'])
      fields = Point1D._field+('y',)
      Point1D = namedtuple('Point2D', fields)
      
      - Optimization - Object Interning In Python - Object interning is a technique used in Python to optimize memory usage and improve performance by reusing immutable objects instead of creating new instances. It is particularly useful for strings, integers and user-defined objects. By interning objects, Python can store only one copy of each distinct object in memory reducing memory consumption and speeding up operations that rely on object comparisons. - Syntax:
      import sys
      interned_string = sys.intern(string_to_intern)
      interned_string2 = sys.intern(string_to_intern)
      interned_string is interned_string2 # faster than str cmp
      
      - All identifiers are interned - Peephole - In Peephole optimization, Python optimizes code either by pre-calculating constant expressions or by membership tests (converting mutable data structures to immutable data structures.) - Peephole Optimization in Python - Find what is pre-calculation by using your_obj.__code__.co_consts - - - - -

Python Fundamental

string.strip()

### Usage 1. remove all empty 2. it allows arguments e.g. \'abbabbacdef\'.strip(\'ab\'). it remove both `a` and `b` (the sequence does not matter) ### exampls
string.strip("   aaa \n ")

calculate total of 1\~100 #card

### Usage 1. sum and range ### exampls
sum(range(1, 100))

sum(range(1, 100))

remove 'key' from dict #card

del dict['key']

merge two dict, dict1 and dict2 #card

dict1.update(dict2)

deduplicate from a list #card

use set

list(set(mylist)) **

*args and **kwargs #card

args: positional arguments

kwargs: kv arguments (dict)

What is dectorator #card

a function return function and the argument is a function

difference between __init__ and __new__ #card

__init__ 是初始化方法,创建对象后,就立刻被默认调用了,可接收参数, first arg is self

__new__ take cls which is current class. It need to return the new class

with #card

it implement finally: file.close()

[1,2,3,4,5] -> [1,4,9,16,25] #card

list comprehension: [i*i for i in l]

map: map(lambda x: x*x, [1,2,3,4,5])

s='abccba' deduplicate and sort #card

''.join(sorted(list(set(s)))

What is difference between remove , discard, del and pop when remove a element from a list #card

remove(v) remove first element with value v

del l[2] remove element from l at pos 2

pop(idx=-1)

discard does not raise exception when key not existed

Sort a dict based on key #card

sorted(dict.items(), key=lambda i: i[0], reverse=False)

find all odd number in a list #card

list comprehension [i for i in l if i%2==1]

filter(): filter(lambda i i%2==1, [1,2,3,4])

merge two list l1, l2 and sort #card

l1.extend(l2); sorted(l1)

a=[[1, 2], [3, 4]] -> [1,2,3,4] #card

two steps [ j for i in a for j in i ] ~for i in a ~ each i is [1, 2], [3, 4]; for j in i -> 1,2 3, 4

**

nvim trick

Some Neovim tricks

\< C-Y > scroll-up

\<C-U> scroll-down

H, M, L go to high/middle/low it should move to first non empty line

d33| | moves the cursor to the specified column in the current line

Find replace

& stands for the matched words

press * and do :%s//xxx/ the search is omitted and it is the current selected word

Move around

+ - can also be used to move to + Move to first none empty char of next line

<CR> can move to head of next line

word and WORD

The difference between words and WORDS is that the former only include letters, digits and numbers.

word can be a sequences of spaces. do dw inside a sequence of space and see what you get. w, b, e, ge are word motions.

W move on WORD boudnary. following are WORD helloVimWord(){}, [1,2,3,4,5] etc

i a; there is also I and A insert at begin of line and end of line

Y yank; it is same as yy; P past above

s and S was remapped in hop/leap. s -> cl S -> cc

**

X delete char left of cursor (\<BS> in norm mode)

command range and g / v command

:. current line

:% entire file

`$~last line

*** ~:12` line 12

:12,14s/import/export/g replace from line 12 to 14

:.,.+12 from current line to current line + 12

:.,$ from current line to eof

:g/import/d delete lines include import

:g!/import/d delete lines do NOT include import it same as v command :v/import/d

motion vs / command

all motion can be replace with find / command. e.g dft can also be d/t<cr>

delete

d0 delete to begin of line

use s to delete

  1. :%s/abc//<CR>

  2. it can also write as :%s/abc<CR>

db backward delete word

dvb and dve delete inclusive ; dvge;

das dap

dF* search backward and delete

with surround di* ca* da* etc

dh delete 1 char to the left; d3h delete 3chars to the right

*

#vimg

g0 go to 0; same as gg

ge jump back end of word

g& it similar to macro, a command in Vim that repeats the last substitution (:s) command on all lines in the file. It's equivalent to :%s//~/g, where // reuses the last search pattern, and ~ reuses the last replacement string.

gJ join [cnt] lines J: join with remove indent, also works in visual mode

gq format text

wrap: gj, gk jump inside long lines , g$, g^

g<C-A> in visual mode/range, increase number of each line based on fist value in range. e.g. 1 1 1 1 -> 1 2 3 4

gu|U + text obj capitalization, e.g. gu$ guW, guu whole line, gU3w, gufk (lowcase to letter k)

`g~` similar to gu|U

gv re-select text

g& apply the replace changes to full document. It is helpful when you need to replace in one line and check if replace is correct and apply to full text

*

Search and replace

*%s/.$/&;/* .$ match a none empty line `&~is what was matched

*** ~:v/^\(/s/\)/;`

  1. :v inverse find /^$ is empty line

  2. /s/$/; command of v; replace EOL to ;

\ze: \ze Zero-width ending split the search pattern into two, first half is match can be used later on, e.g. end\ze(if|for) match both endif and endfor but the matched pattern is end so :s/\vend\ze(if|for)/&_/g replace endif to end_if

put=range(1,10) add 1~10 to buffer

<C-R>= e.g. <C-R>=range(1,10) use register with value of expression

indentation

v> indent current line to the right

>G indent from current line the the end of file to the right

=G remove indent from current to eof

>{ indent to right from begin of block to current line

= in visual mode, indent current line (based on context)

>> indent to the right with normal mode

https://www.vimgolf.com/challenges/596dd9ca448256000c000011

https://www.vimgolf.com/challenges/5ba020f91abf2d000951055c

Fugutive diff merge

reference http://vimcasts.org/episodes/fugitive-vim-resolving-merge-conflicts-with-vimdiff/

Gdiff are used to merge diff conflicts. You can also use Gvdiffsplit! It opens

left(the branch you used to work on, or local branch) . The buff is named as fugutive//2

middle the current working file,

right buffer which is the remote branch that you trying to merged into current branch (remote). the buffer named fugutive://3

If the cursor in middle buffer, diffget //2 get changes from left file and use ]c to jump to next merge conflict. and use diffupdate to refresh file

when done, use only to close other windows

You can use diffput 1 to put changes throught. in fugutive, dp default alias to diffput 1

Gwrite!

Diffview

If git in merge status, DiffViewOpen will show the conflicts files

<Leader>co|ct|cb|ca select Ours, Theirs, Base and All or dx choose none

**

dynamo and lock

  • Lock ** Optimistic Locking *** With Optimistic Locking, there is always one attribute in your DynamoDB table that will act as a “/version number./” It can be a nano-id, an integer, or a timestamp. The version number associated with the record must also be sent when clients request data. *** When client modifies the data. The version number present on the client side must be the same as the item’s version number present in the table item. If it is the same, it means that no other user has changed the record, allowing the write to go through. However, if the version numbers are different, it’s likely that another user has already updated the record, causing DynamoDB to reject your write by throwing the exception - ConditionalCheckFailedException. You can retrieve the item again (with newly updated data) and retry your update when this happens. ** Pessimistic Locking *** Pessimistic Locking is another strategy used by DynamoDB to prevent concurrent updates to a particular row. It use DynamoDB Transactions API *** typescript

+BEGIN_SRC typescript

const AWS = require(“aws-sdk”); const dynamoDB = new AWS.DynamoDB.DocumentClient({ region:’us-east-1’ });

await dynamoDB.transactWrite({ TransactItems: [ { Put: { // Write an item to the Cart table Item: { // Actual Item id: ‘777’, count: ‘1’, description: ‘Lorem ipsum…’ } TableName: “Cart”, }, }, { Update: { // Decrement count of items available to buy only if the count is greater than zero ConditionExpression: “#count > 0”, ExpressionAttributeNames: { “#count”: “count” }, ExpressionAttributeValues: { “:value”: 1, }, Key: { id: ‘777-000’, } TableName: “Items”, UpdateExpression: “SET #count = :count - :value”, }, }, ], }).promise();

+END_SRC

  • Benefits of Optimistic Locking **

Setup postgres debugger env with docker

pldebugger setup

pldebugger require recompile with postgresql source code. A little bit hard to setup. Lucky enough, debian provides already compiled version. Strech: version 10 Buster: version 12

FROM postgres:12

MAINTAINER ray@ray-x
ENV PG_MAJOR 12
ENV PG_VERSION 12.3-1.pgdg100+1

# Install the postgresql debugger
RUN apt-get update \
  && apt-get install -y --no-install-recommends \
  postgresql-$PG_MAJOR-pldebugger


EXPOSE 5432

Start the docker and you should see:

pgdbg           |
pgdbg           | PostgreSQL Database directory appears to contain a database; Skipping initialization
pgdbg           |
pgdbg           | 2020-06-11 03:00:46.211 UTC [1] LOG:  starting PostgreSQL 12.3 (Debian 12.3-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
pgdbg           | 2020-06-11 03:00:46.211 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
pgdbg           | 2020-06-11 03:00:46.211 UTC [1] LOG:  listening on IPv6 address "::", port 5432
pgdbg           | 2020-06-11 03:00:46.214 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
pgdbg           | 2020-06-11 03:00:46.290 UTC [26] LOG:  database system was shut down at 2020-06-21 03:00:32 UTC
pgdbg           | 2020-06-11 03:00:46.314 UTC [1] LOG:  database system is ready to accept connections

Notes that the logs began with pgdbg instead of postgres

To debug with dbeaver, install extension :

CREATE EXTENSION pldbgapi;

Install debug extension in dbeaver (if not yet)

Help -> Install new software dbeaver install Search and install debugger Click “ok”, “accept”, “confirm”… to install After restart dbeaver, you should see a debug icon: DebugIcon

Create a demo sql:

CREATE SCHEMA test;
DROP function if exists test.somefunc(var integer);
CREATE FUNCTION test.somefunc(var integer) RETURNS integer AS $$
DECLARE
   quantity integer := 30+var;
BEGIN
   RAISE NOTICE 'Quantity here is %', quantity;      --在这里的数量是30
   quantity := 50;
   --
   -- 创建一个子块
   --
   DECLARE
      quantity integer := 80;
   BEGIN
      RAISE NOTICE 'Quantity here is %', quantity;   --在这里的数量是80
   END;
   RAISE NOTICE 'Quantity here is %', quantity;      --在这里的数量是50
   RETURN quantity;
END;
$$ LANGUAGE plpgsql;

SELECT test.somefunc(12);

Configure a debug session: Specify database, function, aurgument: Debug config

Start debug debug window

vim-clap is a combination of fzf, ctrlp, leaderF, Ag/Ack, nerdtree(in some extends) ......

Check this: Clap

And this: Clap providers: Clap providers

And this: Clap providers

Yes, it also provide a preview window...... Clap preview: Clap preview window

You can replace you fzf commands with vim-clap, e.g. my vimrc:

noremap <leader><s-F> :Clap grep2 ++query=<cword><CR>
cmap <leader><S-F>h :Clap command_history<CR>
noremap <leader>ch :Clap command_history<CR>
noremap <leader>cf :Clap history<CR>


function! s:history(arg)
  let l:query=''
  let l:subcommand=''
  echo a:arg
  if len(a:arg) > 0
    let l:query=' ++query='+a:arg[1]
  endif

  if a:arg[0] == ':'
    let l:subcommand = 'command_history'
    let l:query=trim(a:arg[1:])
  elseif a:arg[0] == '/'
    let l:subcommand = 'search_history'
    let l:query=trim(a:arg[1:])
  else
    let l:subcommand = 'history'
    let l:query=trim(a:arg)
  endif

  if len(l:query) > 1
    let l:query=' ++query=' . l:query
  endif
  exec 'Clap '. l:subcommand . l:query

endfunction

" noremap <c-F>:Clap grep2 ++query=@visual<CR>
noremap <s-T> :Clap tags<CR>
nmap <S-F2> :Clap filer<CR>

command! -bang -nargs=* History call s:history(<q-args>)
command! Files :Clap files
command! Buffers :Clap buffers
command! Tags :Clap proj_tags
command! Buffers :Clap buffers
command! Commits :Clap commits
command! Gdiff :Clap git_diff_files
command! Jumps :Clap jumps
command! Yanks :Clap yanks
command! Windows :Clap windows
command! Ag :Clap grep ++query<cword>
command! Ag2 :Clap grep2 ++query<cword>

So in command mode, when you type History History! History: it will provides similar interface as fzf