0726b95fd4
added docker instructions
2024-01-17 14:15:46 +01:00
21ca5c626f
added environment variable support
2024-01-17 14:03:43 +01:00
7e2161d4d4
no bak
2024-01-17 13:58:21 +01:00
ef4d2dab16
added docker image support
2024-01-17 13:58:09 +01:00
T.v.Dein
3fd75fa53d
refactored out http fetching code into Fetcher{}/fetch.go
2024-01-16 19:27:46 +01:00
T.v.Dein
78e5de61d2
Add HTTP retries and the possibility to ignore image download errors ( #33 )
...
added HTTP retry and --ignoreerrors which ignores image download errors, fix #30
2024-01-16 13:20:16 +01:00
T.v.Dein
f4a9a9895c
Enhancement/http ( #32 )
...
* added HTTP debug logging using `-d` or `DEBUGHTTP=1` (headers only)
2024-01-16 13:20:16 +01:00
T.v.Dein
ac5b0608d8
fix #30 : revert default adnamedir to just use the slug as before ( #31 )
2024-01-16 13:20:16 +01:00
T.v.Dein
239e253057
Make ad directory name tunable, adapt to kleinanzeigen.de site changes
...
Add template for ad name, adapt kleinanzeigen.de changes
2024-01-12 14:56:08 +01:00
cdf58efd45
fixed changes on kleinanzeigen.de:
...
- Meta did not contain condition and category together anymore, they
removed the category. Therefore fetching (that is, validation)
failed.
- Now we extract the condition and category directly.
- On top, category now includes the whole category tree.
- unit tests had to be tweaked for this measure.
2024-01-12 14:11:02 +01:00
110ee17091
fixed utf8
2024-01-12 13:31:24 +01:00
8321d3c343
added template for ad directory, by default include id now
2024-01-12 13:29:59 +01:00
T.v.Dein
56f53bb777
remove duplicate license badge ( #28 )
...
* remove duplicate license badge
* fix badges
2024-01-06 18:07:33 +01:00
T.v.Dein
9e7f9a2821
Merge pull request #27 from TLINDEN/test/add-main-tests
...
Test/add main tests
2024-01-02 12:51:51 +01:00
577f9d983e
portable error check
2024-01-02 12:36:00 +01:00
114f6b16d9
also added coverage report+badge
2024-01-02 12:31:19 +01:00
a06c730fe4
put all tests into main_test.go, more failure mode tests and verify
2024-01-02 12:22:52 +01:00
d8e968ed6d
better error message on 404
2024-01-02 12:22:26 +01:00
5f450e54ea
add commandline main() test units
2024-01-01 20:53:39 +01:00
3e6349cf36
pass a io.Writer to loggers and outputs so we can test the cmdline
2024-01-01 20:53:05 +01:00
T.v.Dein
bf8e074034
Merge pull request #26 from TLINDEN/test/enhancements
...
Enhanced error checking, added more failure tests
2024-01-01 16:27:20 +01:00
6dea8d78ed
added more invalid tests
2024-01-01 16:24:43 +01:00
ea76b98445
upd httpmock+deps
2024-01-01 16:24:33 +01:00
9f688b7692
put ad code into separate file, enhance error checking
2024-01-01 16:24:07 +01:00
T.v.Dein
d8baa34c54
Test/add mock tests ( #24 )
...
* add scrape unit test using httpmock lib
2023-12-29 13:47:18 +01:00
T.v.Dein
c1cbce32e1
fix linter errors ( #23 )
2023-12-23 22:51:50 +01:00
T.v.Dein
dc4d3d7f9c
add ci pipeline ( #22 )
...
Co-authored-by: Thomas von Dein <tom@izb.net >
2023-12-23 22:36:21 +01:00
T.v.Dein
b28f544416
Doc/add prior art ( #21 )
...
* add mor prior art
2023-12-23 22:20:39 +01:00
T.v.Dein
8cdefe457b
added windows screenshots ( #20 )
...
Co-authored-by: Thomas von Dein <tom@vondein.org >
2023-12-23 18:48:10 +01:00
T.v.Dein
1e4b406aa4
Revert "Fix/newline windows ( #18 )" ( #19 )
...
This reverts commit eaf4db6cef
2023-12-23 18:00:51 +01:00
T.v.Dein
eaf4db6cef
Fix/newline windows ( #18 )
...
* fix #17 : use fmt.Println() after stats
* bump version
2023-12-22 18:59:08 +01:00
T.v.Dein
825649bb3b
added screenshots and a section about prior work ( #16 )
2023-12-21 12:25:04 +01:00
T.v.Dein
6aa9c658b6
add doc link ( #15 )
...
Co-authored-by: Thomas von Dein <tom@vondein.org >
2023-12-19 18:36:11 +01:00
T.v.Dein
2c62f9eb17
fix invalid mod load ( #14 )
...
Co-authored-by: Thomas von Dein <tom@vondein.org >
2023-12-19 18:27:20 +01:00
T.v.Dein
bff0ae553e
Bugfixes ( #13 )
...
* several fixes:
- fix #9 + #10 : switched to koanf module and dropped support for HCL
- fix #11 : disabling colors on windows
- fix #12 : fixed race condition in go routine call inside for loop,
images had been downloaded multiple times
- remove hcl support and use toml format (same thing, better parser)
- update documentation and example config on TOML format of config file
- use Config as arg instead of singular args
- use x/errgroup instead of sync.Waitgroup inside image download loop
---------
Co-authored-by: Thomas von Dein <tom@vondein.org >
2023-12-19 18:23:41 +01:00
T.v.Dein
450d44d129
Dev ( #8 )
...
* fixed conf parsing: variables can now be omitted from the config
* fix newlines: use CRLF on windows
* bump version
---------
Co-authored-by: Thomas von Dein <tom@vondein.org >
2023-12-18 20:18:37 +01:00
T.v.Dein
18f7e0fe49
added proper install instructions ( #7 )
...
Co-authored-by: Thomas von Dein <tom@vondein.org >
2023-12-18 09:48:00 +01:00
T.v.Dein
def063afe9
Merge pull request #6 from TLINDEN/dev
2023-12-18 09:23:55 +01:00
f1908f02cb
bump version
2023-12-18 09:23:18 +01:00
4a528ad9d1
fix #5 : add exe extension to built windows binaries
2023-12-18 09:22:08 +01:00
5c1161f227
fix #4 , use filepath.Join to create portable path's
2023-12-18 09:21:26 +01:00
bd9d8fdb2c
fix version finding
2023-12-17 17:53:01 +01:00
T.v.Dein
1ee886c504
Merge pull request #2 from TLINDEN/dev
...
re-orgainzied code a little, using go templates instead format string
2023-12-17 17:49:27 +01:00
f932d7be83
re-orgainzied code a little, using go templates instead format string
2023-12-17 17:32:05 +01:00
T.v.Dein
d7b13e8a9a
Merge pull request #1 from TLINDEN/dev
...
added custom template support, added more ad data, use concurrency
2023-12-16 20:35:18 +01:00
e904ed6687
added custom template support, added more ad data, use concurrency
2023-12-16 20:32:10 +01:00
df6baadc85
better sample config
2023-12-16 00:01:29 +01:00
314315a1c6
fix pod entities => markdown
2023-12-15 18:29:42 +01:00
2e83e68f20
fix logo
2023-12-15 18:02:34 +01:00
b5e51b43c9
add logo
2023-12-15 18:00:41 +01:00
