10689721794

Committed 03 Sep 2024 07:57PM UTC coverage: 78.98% (-7.4%) from 86.4%

Build # 10689721794

Build Type

Pull #10

github

Committed by

web-flow

Commit Message

Merge fd6209350 into 226e5e483

Pull Request Pull Request #10: Add basic functions

Run Details

66 of 90 branches covered (73.33%)

Branch coverage included in aggregate %.

135 of 168 new or added lines in 6 files covered. (80.36%)

2 existing lines in 1 file now uncovered.

321 of 400 relevant lines covered (80.25%)

4.01 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

57.14

/src/dave_data/io/remote.py

1	import logging	5✔
2	import os	5✔
3	from pathlib import Path	5✔
4
5	import requests	5✔
6
7	from dave_data import config as cfg	5✔
8
9
10	def set_proxy():	5✔
NEW 11	proxy = f"{cfg.get('proxy', 'url')}:{cfg.get('proxy', 'port')}"	×
NEW 12	os.environ["http_proxy"] = proxy	×
NEW 13	os.environ["https_proxy"] = proxy	×
14
15
16	def remove_proxy():	5✔
NEW 17	del os.environ["http_proxy"]	×
NEW 18	del os.environ["https_proxy"]	×
19
20
21	def list_server_dir(url, suffix):	5✔
NEW 22	response = requests.get(url, timeout=20)	×
NEW 23	files = []	×
NEW 24	for r in response.iter_lines():	×
NEW 25	if suffix in str(r):	×
NEW 26	files.append(str(r).split('"')[1])	×
NEW 27	return files	×
28
29
30	def download(fn, url, force=False):	5✔
31	"""
32	Download a file from a given url into a specific file if the file does not
33	exist. Use `force=True` to force the download.
34
35	Parameters
36	----------
37	fn : Path or str
38	Filename with the full path, where to store the downloaded file.
39	url : str
40	Full url of the file.
41	force : bool
42	Set to `True` to download the file even if it already exists.
43
44	Examples
45	--------
46	>>> my_url = "https://upload.wikimedia.org/wikipedia/commons/d/d3/Test.pdf"
47	>>> download("filename.pdf", my_url)
48	>>> os.remove("filename.pdf")
49	"""
50	fn = Path(fn)	5✔
51	if cfg.get("proxy", "use_proxy") is True:	5✔
NEW 52	set_proxy()	×
53	if not fn.is_file() or force:	5✔
54	logging.info("Downloading '%s' from %s", fn.name, url)	5✔
55	try:	5✔
56	req = requests.get(url, timeout=20)	5✔
NEW 57	except requests.exceptions.ProxyError:	×
NEW 58	remove_proxy()	×
NEW 59	req = requests.get(url, timeout=20)	×
60
61	with fn.open("wb") as fout:	5✔
62	fout.write(req.content)	5✔
63	logging.info("%s stored in %s.", fn.name, fn.parent)	5✔
64	else:
65	logging.debug("File '%s' exists. Download is skipped.", fn.name)	5✔
66
67
68	if __name__ == "__main__":	5✔
NEW 69	pass	×

DaveFoss / DAVE_data / 10689721794

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous