Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create setup.py and add bot apis #6

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,9 @@ outputs/*
dataset/*
*.pyc
.idea
venv
venv

# package build
build/*
dist/*
group_purchase.egg-info/*
2 changes: 2 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
include README.md
include group_purchase\assets\html_format.css
39 changes: 33 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,45 @@
# JuWeiHui
Currently a very ugly but usable script to covert group purchase results to printable sheets for delivery during the pandemic in Shanghai

Currently a very ugly but usable script to covert group purchase results to printable sheets for delivery during the
pandemic in Shanghai

## How to use
1. Define your own personal community logic in the community_hardcode folder. Import this class in main.py and change the class name.

1. Define your own personal community logic in the community_hardcode folder. Import this class in main.py and change
the class name.
2. Create 2 folders files and outputfiles.
3. Put all the files you what to generate in files folder.
3. Put all the files you what to generate in files folder.
4. Run main.py
5. If your file has some missind data, an error will be thrown. Ask the one with missing data to fill the data here.
6. The latex pdf file will be generated in outputfiles folder

## Other points

This repo is more like a hardcode repo to solve temporary problems of the JuWeiHui package delivery. We might have more types of excel templates, more communities involved. So it will be appreciate if you can help me to make it less hardcode, and more applicable for more situations.
This repo is more like a hardcode repo to solve temporary problems of the JuWeiHui package delivery. We might have more
types of excel templates, more communities involved. So it will be appreciate if you can help me to make it less
hardcode, and more applicable for more situations.

Users of this form might touch personal data of the purchaser. For me I will delete all those data when the pandemic is
done, and won't send any such data to anyone else. Please make similar declairation if you need to use this script.

I would like to add some algorithm-based features to even improve the delivery logic. If you are a specialist in
optimization, please contact me. Wechat id:wgzjack0305

---

# 订单excel处理方法

1. 先指定小区和excel模板,小区用对象(以后这个对象是全局的,生命周期很长),模板用类(模板对象生命周期很短,处理完就销毁了)
2. 指定输入和输出文件名。输入和输出文件名应和需求名有关,输入模板和输出模板会依赖这个。
3. 调用接口处理excel
1. 创建parser实例,一个parser对应一个excel文件
2. 调用parser解析接口,返回订单集合对象和出错单元格坐标
3. 按需生成pdf或提示用户检查原表格数据

Users of this form might touch personal data of the purchaser. For me I will delete all those data when the pandemic is done, and won't send any such data to anyone else. Please make similar declairation if you need to use this script.
获取excel模板接口:`get_excel_parser(name: str) -> Optional[Type[ExcelParserBase]]`

I would like to add some algorithm-based features to even improve the delivery logic. If you are a specialist in optimization, please contact me. Wechat id:wgzjack0305
- 输入:`name`模板的中文名。
- 输出:模板的Parser类。找不到该名称对应的模板时返回None。
- 当前支持模板:
- 群接龙
- 快团团
8 changes: 5 additions & 3 deletions REFACTORING_PLANS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,15 @@
1. ~~解耦小区信息和团购信息~~
- 1.1 ~~创建excel模板机制的抽象。输出不应依赖输入的表格本身,而是依赖表格解析出的数据抽象类。~~
- 1.2 ~~优化小区类,小区类只应承担小区的独有属性。这也方便之后的架构演进。~~
2. 入口脚本优化
- 2.1 预留支持小区定制,预留支持excel模板定制
3. 美化目录结构和代码格式,提升可维护性。
2. ~~入口脚本优化~~
- 2.1 ~~预留支持小区定制,预留支持excel模板定制~~
3. ~~美化目录结构和代码格式,提升可维护性。~~
4. 新增单元测试
5. 记录适配定制开发者手册
## 阶段二 居委会仓和微信bot仓链接
居委会仓和微信bot仓现在依然是两个独立工作的仓。如果有需要,居委会仓的功能可以接入bot。最终架构中,小区数据和小区定制信息由居委会仓管理,微信bot可以通过统一的接口向居委会仓请求数据或触发动作,这可以降低技术人员维护的成本,因为脚本处理动作将由非技术人员从微信上触发。
1. 居委会仓支持pip安装,以module的形式接入bot。涉及的配置的需要支持动态配置。
2. ~~excel模板搞一个管理器,支持通过字符串查找模板~~
## 阶段三 居委会仓和bot归一
架构的最终形态居委会仓演进成数据处理模块,bot仓演进成用户交互模块。是所有数据均由数据处理模块处理,而对用户的输入输出则由用户交互接管。本开源社区持续维护数据模块,不断降低使用成本和部署成本,并对转发模块选型,使用成本最低、效果最好的用户交互方案。

Expand Down
1 change: 1 addition & 0 deletions group_purchase/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
name = "group_purchase"
File renamed without changes.
6 changes: 6 additions & 0 deletions group_purchase/bot/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
from .bot import on_file, update_file

__all__ = [
'on_file',
'update_file'
]
32 changes: 32 additions & 0 deletions group_purchase/bot/bot.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@

from group_purchase.community.parser_mng import get_community_parser
from group_purchase.purchase_deliver.parser_mng import get_excel_parser

import openpyxl

def on_file(input_file_path, output_file_path, community, excel):
excel_parser = get_excel_parser(excel)
community_parser = get_community_parser(community)

parser = excel_parser(open(input_file_path, 'rb'))
result, errors = parser.parse_for_community(community_parser())
if errors:
return f"以下单元格数据无法识别,请检查:\n{errors}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

返回失败和成功的类型都是str,调用者如何分辨这是成功后生成的路径还是错误提示信息呢?

else:
pass

result.print_to_pdf(output_file_path)
return output_file_path


def update_file(input_file_path, updated_data, coordinate):
try:
workbook = openpyxl.load_workbook(open(input_file_path, 'rb'))
sheet = workbook.active
sheet[coordinate].value = updated_data
workbook.save(input_file_path)
return True
except Exception as error:
print(f"Cannot modify file {input_file_path}, error:\n {error}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这条错误信息调用者get不到,这个只在后台打印还是需要用户感知一下?

return False

5 changes: 5 additions & 0 deletions group_purchase/community/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from .jia_yi_shui_an import JiaYiShuiAn

__all__ = [
'JiaYiShuiAn'
]
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@ class CommunityAddress:
小区地址类,building表示楼栋,供表格分组用;full_addr是地址全写,用来输出
"""

def __init__(self, full_addr, building=''):
def __init__(self, full_addr, building='', room=0):
self.full_addr = full_addr
self.building = building
self.room = room

def __str__(self):
return str(self.full_addr)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,16 +1,12 @@
from .CommunityBase import CommunityBase, CommunityAddress
from utils.chinese import replace_location
from .community_base import CommunityBase, CommunityAddress
from group_purchase.utils.chinese import replace_location


class JiaYiShuiAn(CommunityBase):
def __init__(self):
super().__init__('嘉怡水岸')
self.keywords = ['上海市', '上海', '闵行区', '闵行', '紫龙路', '500号', '五百号', '500']

self.SPECIAL_NUMBERS_BUILDING = {
9999: '居委会'
}

# In JiaYiShuiAn, number 7 -36 are houses, with no room number
self.SPECIAL_NUMBERS_HOUSE = {
5899: '商务楼'
Expand All @@ -26,16 +22,21 @@ def parse_address(self, address):
raise ValueError

if len(locations) == 2:
return CommunityAddress(f'{locations[0]}号楼{locations[1]}号', f'{locations[0]}号楼')
if address.__contains__('5899'):
locations.remove(5899)
return CommunityAddress(f'商务楼{locations[0]}',
'商务楼',
int(locations[0]))
return CommunityAddress(f'{locations[0]}号楼{locations[1]}号', f'{locations[0]}号楼', int(locations[1]))
if len(locations) == 1:
if self.SPECIAL_NUMBERS_BUILDING.__contains__(locations[0]):
return CommunityAddress(f'{self.SPECIAL_NUMBERS_BUILDING[locations[0]]}{locations[0]}号',
self.SPECIAL_NUMBERS_BUILDING[locations[0]])
if self.SPECIAL_NUMBERS_HOUSE.__contains__(locations[0]):
return CommunityAddress(f'{self.SPECIAL_NUMBERS_HOUSE[locations[0]]}{locations[0]}号',
self.SPECIAL_NUMBERS_HOUSE[locations[0]])
self.SPECIAL_NUMBERS_HOUSE[locations[0]],
int(locations[0]))
if address.__contains__('商务楼'):
return CommunityAddress(f'商务楼{locations[0]}', '商务楼')
return CommunityAddress(f'商务楼{locations[0]}',
'商务楼',
int(locations[0]))
if address.__contains__('居委会'):
return CommunityAddress('居委会', '居委会')
raise ValueError
12 changes: 12 additions & 0 deletions group_purchase/community/parser_mng.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
from typing import *

from .community_base import CommunityBase
from .jia_yi_shui_an import JiaYiShuiAn

parser_map: Mapping[str, Type[CommunityBase]] = {
'嘉怡水岸': JiaYiShuiAn,
}


def get_community_parser(name: str) -> Optional[Type[CommunityBase]]:
return parser_map.get(name, None)
5 changes: 5 additions & 0 deletions group_purchase/purchase_deliver/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from .parser_mng import get_excel_parser

__all__ = [
'get_excel_parser'
]
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import openpyxl

from .OrderSet import OrderSet
from group_purchase.purchase_deliver.order_set import OrderSet


class ExcelParser:
class ExcelParserBase:
"""
Excel模板Base类。
"""
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
from pathlib import Path

from .Order import Order
from .ExcelParser import ExcelParser
from group_purchase.purchase_deliver.order import Order
from .excel_parser import ExcelParserBase


# 群接龙
class ExcelParser1(ExcelParser):
class JieLongParser(ExcelParserBase):
"""
Excel模板,第一行:收货人 联系电话 订单总金额 支付状态 收货地址 订购数量
后起每行一条记录,最后一行是合计
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
from audioop import add
from pathlib import Path

from .Order import Order
from .ExcelParser import ExcelParser
from group_purchase.purchase_deliver.order import Order
from .excel_parser import ExcelParserBase


# 快团团
class ExcelParser2(ExcelParser):
class KuaiTuanTuanParser(ExcelParserBase):
"""
Excel模板,第一行:
下单人 团员备注 团长备注 跟团号 商品 规格 数量 商品金额
Expand All @@ -15,11 +16,16 @@ class ExcelParser2(ExcelParser):
商品名和excel文件名保持一致
"""

def __init__(self, file):
super().__init__(file)
self.amount_col = None
self.full_address_col = None

def get_header(self):
for i in range(self.sheet.max_column):
if self.sheet.cell(1, i+1).value == "详细地址":
if self.sheet.cell(1, i + 1).value == "详细地址":
self.full_address_col = i
if self.sheet.cell(1, i+1).value == "数量":
if self.sheet.cell(1, i + 1).value == "数量":
self.amount_col = i

def loop_record(self):
Expand Down
22 changes: 22 additions & 0 deletions group_purchase/purchase_deliver/order.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
from typing import *
from collections import defaultdict
from group_purchase.community.community_base import CommunityAddress


class Order:
def __init__(self):
self.buyer_name: str = ''
self.address: Optional[CommunityAddress] = None
self.items: defaultdict[str, int] = defaultdict(int)

def set_buyer(self, buyer: str) -> 'Order':
self.buyer_name = buyer
return self

def set_address(self, address: CommunityAddress) -> 'Order':
self.address = address
return self

def set_item(self, item_name: str, amount: int) -> 'Order':
self.items[item_name] = amount
return self
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from collections import defaultdict
from pathlib import Path

from utils.PdfGen import html_string_to_pdf
from group_purchase.utils.pdf_gen import html_string_to_pdf


def group_orders_by_address(orders):
Expand All @@ -14,17 +14,19 @@ def group_orders_by_address(orders):
def order_set_to_html(orders, title):
ret = ''
groups = group_orders_by_address(orders)
orders = 0
for group in sorted(groups): # TODO: 改进排序算法,支持数字和非数字混合排序,不要用字典序
orders += int(sum(sum(i.items.values()) for i in groups[group]))
ret += f'''<div class='no-break'>
<h1>{group}, 共{int(sum(sum(i.items.values()) for i in groups[group]))}单 {title}</h1>
<h2>{group}, 共{int(sum(sum(i.items.values()) for i in groups[group]))}单 {title}</h2>
<table>
<tr>
<th>需求概述</th><th>地址</th><th>购买物品</th><th>配送完成</th>
<th>需求概述</th><th>楼栋号</th><th>房间号</th><th>购买物品</th><th>配送完成</th>
</tr>
{''.join(
f'<tr><td>{order.address}: {title}</td><td>{order.address}</td>'
f'<tr><td>{order.address}: {title}</td><td>{order.address.building}</td><td>{order.address.room}</td>'
f'<td>{"<br>".join(f"{item} ×{int(amount)}" for item, amount in order.items.items())}</td><td>&nbsp;</td></tr>'
for order in groups[group])}
for order in sorted(groups[group], key=lambda x: x.address.room))}
</table>
</div>'''
return ret
Expand Down
14 changes: 14 additions & 0 deletions group_purchase/purchase_deliver/parser_mng.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
from typing import *

from .excel_parser.excel_parser import ExcelParserBase
from .excel_parser.jielong_parser import JieLongParser
from .excel_parser.kuaituantuan_parser import KuaiTuanTuanParser

parser_map: Mapping[str, Type[ExcelParserBase]] = {
'群接龙': JieLongParser,
'快团团': KuaiTuanTuanParser
}


def get_excel_parser(name: str) -> Optional[Type[ExcelParserBase]]:
return parser_map.get(name, None)
1 change: 1 addition & 0 deletions group_purchase/utils/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__all__ = []
File renamed without changes.
File renamed without changes.
4 changes: 2 additions & 2 deletions src/utils/PdfGen.py → group_purchase/utils/pdf_gen.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
import pdfkit


# TODO: 写的有点草率,之后再斟酌下。。。
def html_string_to_pdf(html, filename):

pdfkit.from_string(html, filename, configuration=pdfkit.configuration(
wkhtmltopdf=r'C:\Program Files\wkhtmltopdf\bin\wkhtmltopdf.exe'), options={
'encoding': 'utf-8',
'user-style-sheet': 'assets/html_format.css',
'user-style-sheet': 'group_purchase/assets/html_format.css',
'footer-center': '第[page]页 共[topage]页'
})
File renamed without changes.
Loading