python 学习笔记之工程化

发表于 更新于 分类于

依赖管理

  1. pip freeze > requirements.txt
  2. pip install pipreqs
    pipreqs ./ --encoding=utf8 --force

log 记录

简单使用

1
2
3
4
5
6
import logging

logging.basicConfig(format='%(levelname)s - %(asctime)s - %(message)s',level=logging.DEBUG)
logging.warning('this is %s log','warning')
logging.info('info log')
logging.error('error log')

运行代码输出：

1
2
3
4
WARNING - 2023-04-21 10:27:38,631 - this is warning log
INFO - 2023-04-21 10:27:38,631 - info log
ERROR - 2023-04-21 10:27:38,631 - error log

yaml 配置 log

采用 YAML 格式，用于新的基于字典的方法

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
version: 1
formatters:
  simple:
    format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
handlers:
  console:
    class: logging.StreamHandler
    level: DEBUG
    formatter: simple
    stream: ext://sys.stdout
loggers:
  app:
    level: DEBUG
    handlers: [console]
    propagate: no
root:
  level: DEBUG
  handlers: [console]

使用如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
import logging.config

import yaml

with open(file='conf/logging.yaml', mode='r', encoding='utf8') as file:
    logging_yaml = yaml.load(stream=file, Loader=yaml.FullLoader)
    logging.config.dictConfig(config=logging_yaml)

logger = logging.getLogger('app')
if __name__ == '__main__':
    logger.warning('this is %s log', 'warning')
    logger.info('info log')
    logger.error('error log')

配置文件

环境变量中配置信息

1
2
3
import os

secret_key = os.environ.get('SECRET_KEY', 'jajajajhh')

yaml 配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
class Configure:
    def __init__(self):
        self.map_parameters = {}

        self.cfg_file = os.path.abspath(os.path.join(os.path.dirname(__file__), ".conf/conf.yaml"))
        self.init_local()

    def init_local(self):
        if os.path.exists(self.cfg_file):
            with open(self.cfg_file, "r+", encoding="utf-8") as reader:
                configure = yaml.load(reader, Loader=yaml.SafeLoader)
                if "hbase" not in configure.keys():
                    raise ValueError("not  hbase tables configure")
                self.map_parameters["hbase"] = configure["hbase"]
                self.map_parameters["hbase.common_param"] = configure["hbase"]["common_param"]
                self.map_parameters["hbase.tables"] = configure["hbase"]["tables"]

使用如下：

1
2
conf = Configure()
hbase_param = conf.map_parameters["hbase"]