🎉 New Source: Awin #21116

leo-schick · 2023-01-06T16:43:36Z

What

Add new source for AWIN. This version implements only the Advertiser API which is probably used by most users. Streams of the Publisher API which use the same API endpoint are - except the accounts stream - not implemented.

Implemented streams:

accounts
publishers (of the advertiser accounts)
advertiser_transactions

How

Using the Python SDK / HTTP API source following the Python CDK: Creating a HTTP API Source tutorial.

It could probably easily be rebuild using the low-code SDK but I'd liked the Python SDK more since it gives greater flexibility :-)

Pre-merge Checklist

Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub by running the /publish command described here
After the connector is published, connector added to connector index as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Tests

Unit

Integration

Put your integration tests output here.

Acceptance

Acceptance test result:

CLAassistant · 2023-01-06T16:43:42Z

All committers have signed the CLA.

leo-schick · 2023-01-09T16:32:19Z

Note: Incremental sync. for stream advertiser_transactions is implemented but does not work yet. Full sync. works.

implemented streams: - accounts - publishers (of the advertiser accounts) - advertiser_transactions For the API see https://wiki.awin.com/index.php/Advertiser_API

leo-schick · 2023-01-30T16:01:41Z

gradlew - airbytePythonFormat

Result of ./gradlew :airbyte-integrations:connectors:source-awin:airbytePythonFormat:

(.venv) *[source-awin][~/git/leo-schick/airbyte]$ ./gradlew :airbyte-integrations:connectors:source-awin:airbytePythonFormat
Building all of Airbyte.
/srv/git/leo-schick/airbyte/airbyte-integrations/connectors

> Configure project :
configuring docker task for airbyte-bootloader
configuring docker task for airbyte-connector-builder-server
configuring docker task for airbyte-container-orchestrator
configuring docker task for airbyte-cron
configuring docker task for airbyte-proxy
configuring docker task for airbyte-server
configuring docker task for airbyte-temporal
configuring docker task for airbyte-webapp
configuring docker task for airbyte-workers
configuring docker task for init
configuring docker task for db-lib
configuring docker task for reporter

> Task :airbyte-integrations:connectors:source-awin:checkPython
Using python 3.9.2 from /srv/git/leo-schick/airbyte/airbyte-integrations/connectors/source-awin/.venv (.venv/bin/python)
Using pip 21.3.1 from /srv/git/leo-schick/airbyte/airbyte-integrations/connectors/source-awin/.venv/lib/python3.9/site-packages/pip (python 3.9)

> Task :airbyte-integrations:connectors:source-awin:pipInstall
[python] .venv/bin/python -m pip install pip==21.3.1
         Requirement already satisfied: pip==21.3.1 in ./.venv/lib/python3.9/site-packages (21.3.1)
         WARNING: You are using pip version 21.3.1; however, version 22.3.1 is available.
         You should consider upgrading via the '/srv/git/leo-schick/airbyte/airbyte-integrations/connectors/source-awin/.venv/bin/python -m pip install --upgrade pip' command.
[python] .venv/bin/python -m pip list --format=columns
         Package                Version   Editable project location
         ---------------------- --------- -----------------------------------------------------------------------------
         airbyte-cdk            0.18.1
         appdirs                1.4.4
         attrs                  22.2.0
         backoff                2.2.1
         black                  22.3.0
         cachetools             5.2.1
         cattrs                 22.2.0
         certifi                2022.12.7
         charset-normalizer     2.1.1
         click                  8.1.3
         coverage               6.3.1
         dataclasses-jsonschema 2.15.1
         deepdiff               5.8.1
         Deprecated             1.2.13
         docker                 5.0.3
         dpath                  2.0.8
         exceptiongroup         1.1.0
         fancycompleter         0.9.1
         flake8                 4.0.1
         genson                 1.2.2
         hypothesis             6.54.6
         hypothesis-jsonschema  0.20.1
         icdiff                 1.9.1
         idna                   3.4
         inflection             0.5.1
         iniconfig              2.0.0
         isodate                0.6.1
         isort                  5.6.4
         Jinja2                 3.1.2
         jsonref                0.2
         jsonschema             3.2.0
         MarkupSafe             2.1.1
         mccabe                 0.6.1
         mypy                   0.930
         mypy-extensions        0.4.3
         ordered-set            4.1.0
         packaging              23.0
         pathspec               0.11.0
         pdbpp                  0.10.3
         pendulum               2.1.2
         pip                    21.3.1
         pkg_resources          0.0.0
         platformdirs           2.6.2
         pluggy                 1.0.0
         pprintpp               0.4.0
         py                     1.11.0
         pycodestyle            2.8.0
         pydantic               1.9.2
         pyflakes               2.4.0
         Pygments               2.14.0
         pyproject-flake8       0.0.1a2
         pyrepl                 0.9.0
         pyrsistent             0.19.3
         pytest                 6.2.5
         pytest-cov             3.0.0
         pytest-mock            3.6.1
         pytest-sugar           0.9.6
         pytest-timeout         1.4.2
         python-dateutil        2.8.2
         pytzdata               2020.1
         PyYAML                 5.4.1
         requests               2.28.1
         requests-cache         0.9.7
         requests-mock          1.9.3
         setuptools             44.1.1
         six                    1.16.0
         sortedcontainers       2.4.0
         source-acceptance-test 0.0.0     /srv/git/leo-schick/airbyte/airbyte-integrations/bases/source-acceptance-test
         source-awin            0.0.0     /srv/git/leo-schick/airbyte/airbyte-integrations/connectors/source-awin
         termcolor              2.2.0
         toml                   0.10.2
         tomli                  2.0.1
         typing_extensions      4.4.0
         url-normalize          1.4.3
         urllib3                1.26.13
         websocket-client       1.4.2
         wmctrl                 0.4
         wrapt                  1.14.1
         WARNING: You are using pip version 21.3.1; however, version 22.3.1 is available.
         You should consider upgrading via the '/srv/git/leo-schick/airbyte/airbyte-integrations/connectors/source-awin/.venv/bin/python -m pip install --upgrade pip' command.

> Task :airbyte-integrations:connectors:source-awin:isortFormat
[python] .venv/bin/python -m isort --settings-file=/srv/git/leo-schick/airbyte/pyproject.toml ./
         Skipped 7 files

> Task :airbyte-integrations:connectors:source-awin:blackFormat
[python] .venv/bin/python -m black --config /srv/git/leo-schick/airbyte/pyproject.toml ./
         All done! ✨ 🍰 ✨
         5 files left unchanged.

> Task :airbyte-integrations:connectors:source-awin:flakeCheck
[python] .venv/bin/python -m pflake8 --config /srv/git/leo-schick/airbyte/pyproject.toml ./

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.6/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 16s
16 actionable tasks: 5 executed, 11 up-to-date
(.venv) *[source-awin][~/git/leo-schick/airbyte]$

leo-schick · 2023-01-30T16:06:59Z

rebased on v0.40.30

marcosmarxm · 2023-06-02T18:05:36Z

Hello, our team ended up losing track of reviewing your contribution and we're sorry about that. I'm cleaning up our backlog and closing it now. If you'd like to resubmit it, we'll be very happy and we'll do our best to prioritize the proper review. For new connectors, the project always aims to obtain integration accounts with the implemented service to ensure that the connector works effectively over time. Once again, I apologize, and feel free to send me a message on Slack if you have any further questions.

octavia-squidington-iv added area/connectors Connector related issues community connectors/source/awin labels Jan 6, 2023

leo-schick added 5 commits January 30, 2023 14:14

add source-awin first version 0.1.0

cfea5e0

implemented streams: - accounts - publishers (of the advertiser accounts) - advertiser_transactions For the API see https://wiki.awin.com/index.php/Advertiser_API

fix stream advertiser_transactions first incremental sync.

bfdbb37

mark oauth2 token with airbyte_secret

296a39e

improve docs

f48aad2

fixes to make acceptance tests work

294fd9f

leo-schick force-pushed the source-awin branch from 58620e5 to 294fd9f Compare January 30, 2023 15:52

octavia-squidington-iv added the area/documentation Improvements or additions to documentation label Jan 30, 2023

improve formatting

bf866a4

leo-schick marked this pull request as ready for review January 30, 2023 16:05

octavia-squidington-iv added the bounty label Jan 30, 2023

marcosmarxm added the new-source label Apr 14, 2023

marcosmarxm added community-legacy and removed community labels May 15, 2023

marcosmarxm closed this Jun 2, 2023

leo-schick mentioned this pull request Jul 26, 2023

🎉 New Source: Awin #28728

Closed

marcosmarxm added community and removed community-legacy labels Aug 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

🎉 New Source: Awin #21116

🎉 New Source: Awin #21116

Uh oh!

leo-schick commented Jan 6, 2023 •

edited

Loading

Uh oh!

CLAassistant commented Jan 6, 2023 •

edited

Loading

Uh oh!

leo-schick commented Jan 9, 2023

Uh oh!

leo-schick commented Jan 30, 2023 •

edited

Loading

Uh oh!

leo-schick commented Jan 30, 2023

Uh oh!

marcosmarxm commented Jun 2, 2023

Uh oh!

Uh oh!

🎉 New Source: Awin #21116

🎉 New Source: Awin #21116

Uh oh!

Conversation

leo-schick commented Jan 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

Pre-merge Checklist

Community member or Airbyter

Airbyter

Tests

Uh oh!

CLAassistant commented Jan 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leo-schick commented Jan 9, 2023

Uh oh!

leo-schick commented Jan 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

gradlew - airbytePythonFormat

Uh oh!

leo-schick commented Jan 30, 2023

Uh oh!

marcosmarxm commented Jun 2, 2023

Uh oh!

Uh oh!

leo-schick commented Jan 6, 2023 •

edited

Loading

CLAassistant commented Jan 6, 2023 •

edited

Loading

leo-schick commented Jan 30, 2023 •

edited

Loading