nuttycom/aftok - Change EW2XN7KUMCAQNVFJJ5YTAVDZCPHNWDOEDMRFBUGLY6IE2HKNNX5AC

Update docker build, clean up migration for payments tables.

Created by Kris Nuttycombe on March 2, 2017

EW2XN7KUMCAQNVFJJ5YTAVDZCPHNWDOEDMRFBUGLY6IE2HKNNX5AC

Dependencies

In channels

main

Change contents

File deletion: notes.md

BF:BFD[5.2] → [6.19097:19129]

BF:BFD[6.19129] → [6.17955:17955]

B:BD[6.17955] → [6.17956:19096]

B:BD[6.19096] → [7.1:4498]

Aftok
=====
> *"I have to figure out how to balance the work I can make a living on with the 
> work I can’t, because the work I can’t make a living on is more important."*
>     -- [Meredith L. Patterson](https://medium.com/message/how-i-explained-heartbleed-to-my-therapist-4c1dbcbe1099)
It's impossible to make a living writing open-source software. Certainly, there
are exceptions; a few companies eke out a relatively spare existence selling
"enterprise support" licenses for free software, and lots of people employed by
private companies contribute to open-source products on time paid for by their
employer. Nonetheless, a great deal of the open-source software that developed
is a labor of love. OSS developers are artists, but they're fortunate that
their art is in a medium that also, at present, makes it easy enough to find
paid work that few of us have to starve while making our art.
Unlike many other arts (excepting music) open-source software development is 
frequently a deeply collaborative endeavor. 
* Trust cannot be imposed, it can only be grown.
* Last section needs to be more positive, deemphasize risk.
(12:36:54 PM) nuttycom1: Reiterating my earlier question, this time via StackOverflow: http://stackoverflow.com/questions/30466275/http-basic-auth-in-snap
(12:41:03 PM) dmj`: nuttycom1: why not use snap's cookie-based auth?
(12:41:33 PM) carter_cloud: mightybyte:  lpsmith  did that posgres-simple snaplet transaction bug ever get fixed?
(12:41:40 PM) nuttycom1: dmj`: my clients aren't necessarily browsers.
(12:41:55 PM) nuttycom1: In fact, they're mostly not browsers.
(12:43:17 PM) dmj`: nuttycom1: are they mobile clients?
(12:43:31 PM) nuttycom1: dmj`: no, other servers.
(12:48:21 PM) dmj`: nuttycom1: basic auth isn't secure, you're sending the password (base-64 encoded) on every request, it could be fine if it's all over https. But still, a token based system would be more secure. Are these severs internal / external? (i.e. yours or someone elses)
(12:53:16 PM) nuttycom1: dmj`: Token-based systems are no more secure if they're not over https, but that's a side point; in my experience it's common in any case to use the Authorization headers to carry authentication tokens so that they don't have to pollute the request payload in other ways. 
(12:54:14 PM) nuttycom1: I have a bunch of CLI tools that access my services via curl and such, which prompt for auth credentials along the way. Using Basic is convenient for these kinds of tools.
(12:54:49 PM) nuttycom1: All the servers *are* in my control, however. I suppose I could use an X-My-Auth header or some such, but I don't much see the point.
(12:55:38 PM) nuttycom1: At least, they're under my control at the moment, but obviously the API I'm providing is ultimately going to be accessible to anyone on the open web, so I might as well make things easy for them.
(12:59:00 PM) mightybyte: carter_cloud: Yes
(12:59:32 PM) mightybyte: http://blog.melding-monads.com/2015/02/12/announcing-snaplet-postgresql-simple-0-6/
(01:00:42 PM) dmj`: nuttycom1: token based systems only transmit the credentials on the initialize request to generate the token, not on every request, so not sure how it's just as secure. It seems like setting up your own oauth provider would be ideal in this case, especially if you plan on releasing this api to the public. If your servers are not exposed to the outside world, why even have them authenticate against each other, if you're using A
(01:00:43 PM) dmj`: you using a VPC? 
(01:00:49 PM) dmj`: initial*
(01:00:52 PM) mightybyte: carter_cloud: I guess that post never hit reddit.
(01:02:17 PM) mightybyte: nuttycom1: I don't know of a basic auth package for snap (probably in part due to the concerns dmj` has pointed out), but I wouldn't expect that it would be too hard to write.
(01:02:42 PM) nuttycom1: mightybyte: Hah.... yeah, I provide one as context in my SO question.
(01:04:49 PM) carter_cloud: mightybyte: btw Stephen Diehl has a neat prototype of a streaming Postgres binding https://github.com/elsen-trading/pgstream
(01:07:49 PM) nuttycom1: dmj`: fair enough; I suppose that I can add token handling to the CLI tools, it's just a little more work since I'll have to encrypt the token before it's stored locally, and require the user to decrypt on each request rather than provide credentials. That's probably the right way to go.
(01:07:51 PM) dmj`: nuttycom1: I can't speak for other token-based systems, but json web tokens are typically encrpyted (HMAC SHA-256), basic auth data is just encoded. So your last line of defense in basic auth is ssl, and your surface area (every request) is much larger. I dunno, if you think basic auth suits your needs then go for it, but it sounds to me like you should become an oauth provider here.
(01:08:16 PM) mightybyte: nuttycom1: responded
(01:09:10 PM) nuttycom1: mightybyte: thanks!
(01:10:45 PM) dmj`: nuttycom1: yea, you can create your own key server! Then create tcp connections from your key server to all other api servers. You should change the key somewhat frequently (This secret key is used to hash all tokens). So your key server can broadcast the new secret to all servers via tcp, then (if you're using haskell and not that node.js single-threaded stuff) you could migrate everyones tokens over to the new key transparen
(01:10:45 PM) dmj`: if new key has been sent, then rehash with it, and put it in the header).
(01:11:16 PM) dmj`: only one-thread, I don't know how people do it
(01:11:20 PM) dmj`: one thread*
(01:12:18 PM) nuttycom1: Interesting, thanks dmj`. Looks like I have some additional investigation to do.

Insertion in .dockerignore at line 1 [8.1]
[8.1]
[8.2]
```
.git
.gitignore
.stack-work
```

Insertion in .dockerignore at line 5 [8.1]

[8.3]

local
migrations
work
Makefile
Dockerfile
./*.md
./*.cfg

Insertion in .gitignore at line 19 [11.135]
[9.7]
[10.1]
```
bin
```

Replacement in Dockerfile at line 14 [8.43]

B:BD[12.407] → [13.232:295]

    apt-get install -y --no-install-recommends libpq-dev stack

[12.407]

[14.1]

    apt-get install -y --no-install-recommends \
    build-essential autotools-dev autoconf dh-autoreconf \
    libpq-dev libsqlite3-dev \
    git stack

Replacement in Dockerfile at line 19 [8.43]

B:BD[14.2] → [14.2:59]

# Set up /etc/aftok volume for configuration information

[14.2]

[14.59]

# Set up /etc/aftok volume for mounting configuration from the host system

Deletion in Dockerfile at line 22 [8.43]
B:BD[14.102] → [14.102:152]
```
ADD ./conf/aftok.cfg.example /etc/aftok/aftok.cfg
```

Insertion in Dockerfile at line 31 [8.43]

[15.67]

[16.30]


# Install ghc globally so that we don't have to reinstall it
# whenever we change stack.yaml or aftok.cabal
RUN stack --resolver lts-7.16 setup

Replacement in Dockerfile at line 37 [8.43]

B:BD[16.71] → [16.71:118]

ADD ./docker/stack.yaml  /opt/aftok/stack.yaml

[16.71]

[13.440]

ADD ./stack.yaml  /opt/aftok/stack.yaml

Deletion in Dockerfile at line 44 [8.43]
B:BD[3.1142] → [3.1142:1211]
∅:D[15.136] → [8.1024:1025]
∅:D[14.368] → [8.1024:1025]
∅:D[13.482] → [8.1024:1025]
∅:D[12.871] → [8.1024:1025]
∅:D[3.1211] → [8.1024:1025]
B:BD[8.1024] → [8.1024:1025]
```
RUN apt-get install -y libsqlite3-dev
RUN stack install dbmigrations
```
Deletion in Dockerfile at line 47 [8.43]
B:BD[16.222] → [3.1212:1252]
```
ADD ./migrations  /opt/aftok/migrations
```
Replacement in Makefile at line 2 [3.1254]
B:BD[3.1272] → [3.1272:1310]
```
	docker build -t aftok/aftok:latest .
```
[3.1272]
[3.1310]
```
	sudo docker build -t aftok/aftok:latest .
```

Replacement in Makefile at line 4 [3.1254]

B:BD[3.1311] → [3.1311:1432]

run-local-docker:
	docker run --net=host -it -v /home/nuttycom/projects/aftok/docker-conf/:/etc/aftok aftok/aftok:latest

[3.1311]

[3.1432]

run-local-docker: build-container
	sudo docker run --net=host -it -v /home/nuttycom/projects/aftok/local/conf/:/etc/aftok aftok/aftok:latest

File addition: bin (d--x------)
[5.2]
File addition: .keep (----------)
[0.703]

File deletion: stack.yaml

BF:BFD[8.1510] → [16.496:530]

BF:BFD[16.530] → [16.319:319]

B:BD[16.319] → [16.320:495]

flags: {}
packages:
- '.'
extra-deps: 
- snaplet-postgresql-simple-0.6.0.4
- resource-pool-catchio-0.2.1.0
resolver: lts-5.3
#allow-newer: true
local-bin-path: /opt/aftok/bin

File addition: docs (d--x------)
[5.2]
File move: README.md → design.md
BF:BFD[5.2] → [17.5048:5081]
BF:BF[17.5081] → [17.1:1]
[0.754]
[17.1]
File move: features.md → features.md
BF:BFD[5.2] → [6.17918:17953]
BF:BF[6.17953] → [6.17292:17292]
[0.754]
[6.17292]
File move: inception.md → inception.md
BF:BFD[5.2] → [2.1:37]
BF:BF[2.37] → [5.4:4]
[0.754]
[5.4]
File move: MAYBE.md → maybe.md
BF:BFD[5.2] → [18.1530:1562]
BF:BF[18.1562] → [18.1:1]
[0.754]
[18.1]
File move: TASKS.md → tasks.md
BF:BFD[5.2] → [19.3395:3427]
BF:BF[19.3427] → [19.1:1]
[0.754]
[19.1]

Replacement in lib/Aftok/Database/PostgreSQL.hs at line 218 [22.456]

B:BD[20.264] → [21.392:434]

B:BD[21.434] → [23.533:623]

storeEvent (CreateSubscription uid bid) =
  Just $ storeEventJSON (Just uid) "create_subscription" (createSubscriptionJSON uid bid)

[20.264]

[20.384]

storeEvent (CreateSubscription uid bid t) =
  Just $ storeEventJSON (Just uid) "create_subscription" (createSubscriptionJSON uid bid t)

Replacement in lib/Aftok/Database/PostgreSQL.hs at line 438 [22.456]

B:BD[24.12002] → [25.403:522]

    \(project_id, event_id, name, description, recurrence_type, recurrence_count, billing_amount, grace_period_days) \

[24.12002]

[24.12120]

    \( project_id, event_id, name, description \
    \, recurrence_type, recurrence_count \
    \, billing_amount, grace_period_days) \

Replacement in lib/Aftok/Database/PostgreSQL.hs at line 460 [22.456]
B:BD[26.3943] → [21.518:566]
```
pgEval dbop @ (CreateSubscription uid bid) = do
```
[26.3943]
[25.576]
```
pgEval dbop @ (CreateSubscription uid bid start_date) = do
```
Replacement in lib/Aftok/Database/PostgreSQL.hs at line 465 [22.456]
B:BD[25.710] → [25.710:746]
```
    \VALUES (?, ?, ?) RETURNING id"
```
[25.710]
[21.567]
```
    \VALUES (?, ?, ?, ?) RETURNING id"
```
Insertion in lib/Aftok/Database/PostgreSQL.hs at line 469 [22.456]
[21.647]
[21.647]
```
    , fromThyme start_date
```
Insertion in lib/Aftok/Database.hs at line 14 [28.4936]
[27.1893]
[22.6576]
```
import           Data.Thyme.Time           as T (Day)
```

Replacement in lib/Aftok/Database.hs at line 56 [28.4936]

B:BD[24.13383] → [21.1644:1712]

  CreateSubscription :: UserId -> BillableId -> DBOp SubscriptionId

[24.13383]

[21.1712]

  CreateSubscription :: UserId -> BillableId -> T.Day -> DBOp SubscriptionId

Replacement in lib/Aftok/Json.hs at line 9 [29.19]
B:BD[29.156] → [30.918:949]
```
import           ClassyPrelude
```
[29.156]
[31.1965]
```
import           ClassyPrelude                    hiding (Day)
```
Insertion in lib/Aftok/Json.hs at line 26 [29.19]
[32.58]
[33.3094]
```
import           Data.Thyme.Time                  (Day)
```

Replacement in lib/Aftok/Json.hs at line 105 [29.19]

B:BD[21.2437] → [21.2437:2523]

idJSON :: forall a. Lens' a UUID -> a -> Value
idJSON l a = toJSON . tshow $ view l a

[21.2437]

[29.485]

idValue :: forall a. Lens' a UUID -> a -> Value
idValue l a = toJSON . tshow $ view l a
idJSON :: forall a. Text -> Lens' a UUID -> a -> Value
idJSON t l a  = v1 $ obj [ t .=  idValue l a ]

Replacement in lib/Aftok/Json.hs at line 113 [29.19]
B:BD[34.6666] → [16.1197:1247]
```
  obj [ "projectId" .=  tshow (pid ^. _ProjectId)
```
[34.6666]
[16.1247]
```
  obj [ "projectId" .=  idValue _ProjectId pid
```
Replacement in lib/Aftok/Json.hs at line 118 [29.19]
B:BD[35.286] → [35.286:311]
B:BD[35.311] → [16.1297:1348]
```
projectIdJSON pid = v1 $
  obj [ "projectId" .= tshow (pid ^. _ProjectId) ]
```
[35.286]
[36.1693]
```
projectIdJSON = idJSON "projectId" _ProjectId
```
Replacement in lib/Aftok/Json.hs at line 128 [29.19]
B:BD[35.402] → [35.402:427]
B:BD[35.427] → [16.1517:1568]
```
auctionIdJSON pid = v1 $
  obj [ "auctionId" .= tshow (pid ^. _AuctionId) ]
```
[35.402]
[35.481]
```
auctionIdJSON = idJSON "auctionId" _AuctionId
```

Replacement in lib/Aftok/Json.hs at line 132 [29.19]

B:BD[37.592] → [16.1569:1694]

  obj [ "projectId"    .= tshow (x ^. (A.projectId._ProjectId))
      , "initiator"    .= tshow (x ^. (A.initiator._UserId))

[37.592]

[26.5070]

  obj [ "projectId"    .= idValue (A.projectId._ProjectId) x
      , "initiator"    .= idValue (A.initiator._UserId) x

Replacement in lib/Aftok/Json.hs at line 143 [29.19]

B:BD[16.1894] → [16.1894:1989]

creditToJSON (CreditToUser uid)     = v2 $ obj [ "creditToUser"    .= tshow (uid ^. _UserId) ]

[16.1894]

[16.1989]

creditToJSON (CreditToUser uid)     = v2 $ obj [ "creditToUser"    .= idValue _UserId uid ]

Replacement in lib/Aftok/Json.hs at line 163 [29.19]
B:BD[36.2094] → [38.5450:5483]
B:BD[38.5483] → [16.2202:2235]
∅:D[16.2235] → [39.1798:1799]
∅:D[38.5519] → [39.1798:1799]
B:BD[36.2133] → [39.1798:1799]
```
eventIdJSON (EventId eid) = v1 $
  obj [ "eventId" .= tshow eid ]
```
[36.2094]
[40.130]
```
eventIdJSON = idJSON "eventId" _EventId
```
Replacement in lib/Aftok/Json.hs at line 176 [29.19]
B:BD[38.5600] → [38.5600:5641]
B:BD[38.5641] → [16.2343:2380]
```
amendmentIdJSON (AmendmentId aid) = v1 $
  obj [ "amendmentId" .= tshow aid ]
```
[38.5600]
[20.1568]
```
amendmentIdJSON = idJSON "amendmentId" _AmendmentId
```

Replacement in lib/Aftok/Json.hs at line 183 [29.19]

B:BD[21.2627] → [21.2627:2691]

  [ "projectId"   .= (b ^. (B.project . _ProjectId . to tshow))

[21.2627]

[21.2691]

  [ "projectId"   .= idValue (B.project . _ProjectId) b

Replacement in lib/Aftok/Json.hs at line 199 [29.19]

B:BD[38.5682] → [21.2998:3188]

createSubscriptionJSON :: UserId -> B.BillableId -> Value
createSubscriptionJSON uid bid = v1 $
  obj [ "user_id"     .= idJSON _UserId uid
      , "billable_id" .= idJSON B._BillableId bid

[38.5682]

[41.621]

createSubscriptionJSON :: UserId -> B.BillableId -> Day -> Value
createSubscriptionJSON uid bid d = v1 $
  obj [ "user_id"     .= idValue _UserId uid
      , "billable_id" .= idValue B._BillableId bid
      , "start_date"  .= showGregorian d

Replacement in lib/Aftok/Json.hs at line 211 [29.19]

B:BD[21.3359] → [21.3359:3475]

  [ "user_id"     .= idJSON (B.customer . _UserId) sub
  , "billable_id" .= idJSON (B.billable . B._BillableId) sub

[21.3359]

[21.3475]

  [ "user_id"     .= idValue (B.customer . _UserId) sub
  , "billable_id" .= idValue (B.billable . B._BillableId) sub

Replacement in lib/Aftok/Json.hs at line 222 [29.19]

B:BD[21.3694] → [4.396:601]

  [ "subscription_id" .=
     view (subscription . B._SubscriptionId . to tshow) r
  , "payment_request_protobuf_64" .=
     view (paymentRequest . to (decodeUtf8 . B64.encode . runPut . encodeMessage)) r

[21.3694]

[4.601]

  [ "subscription_id" .= idValue (subscription . B._SubscriptionId) r
  , "payment_request_protobuf_64" .= view prBytes r

Insertion in lib/Aftok/Json.hs at line 228 [29.19]

[21.4011]

  where 
    prBytes = (paymentRequest . to (decodeUtf8 . B64.encode . runPut . encodeMessage))

Replacement in lib/Aftok/Json.hs at line 246 [29.19]

B:BD[41.1029] → [41.1029:1216]

  obj [ "payment_request_id"  .= (r ^. (request . _PaymentRequestId . to tshow))
      , "payment_protobuf_64" .= (r ^. (payment . to (decodeUtf8 . B64.encode . runPut . encodeMessage)))

[41.1029]

[41.1216]

  obj [ "payment_request_id"  .= idValue (request . _PaymentRequestId) r
      , "payment_protobuf_64" .= view paymentBytes r

Insertion in lib/Aftok/Json.hs at line 250 [29.19]

[20.2430]

  where 
    paymentBytes = payment . to (decodeUtf8 . B64.encode . runPut . encodeMessage)

Insertion in lib/Aftok/Json.hs at line 337 [29.19]

[16.4874]

[40.987]


parseBillable :: Value -> Parser B.Billable
parseBillable = unversion "Billable" p where
  --p (Version 1 0) o = 
  p v o = badVersion "Billable" v o

Insertion in lib/Aftok/Json.hs at line 344 [29.19]
[40.988]

Replacement in migrations/2016-12-31_03-45-17_create-payments.txt at line 58 [26.7446]

B:BD[26.9257] → [26.9257:9289]

    request_data bytea not null

[26.9257]

[26.9289]

    request_data bytea not null,
    url_key text not null,
    request_time timestamp with time zone not null,
    billing_date date not null

Replacement in migrations/2016-12-31_03-45-17_create-payments.txt at line 68 [26.7446]

B:BD[26.9513] → [26.9513:9545]

    payment_data bytea not null

[26.9513]

[26.9545]

    payment_data bytea not null,
    payment_date timestamp with time zone not null,
    exchange_rates json

Insertion in migrations/2016-12-31_03-45-17_create-payments.txt at line 78 [26.7446]
[26.9667]
[26.9667]
```
  drop type recurrence_t;
```
Insertion in migrations/2016-12-31_03-45-17_create-payments.txt at line 80 [26.7446]
[26.9694]
[26.9694]
```
  drop type aftok_event_t;
```
Insertion in migrations/2016-12-31_03-45-17_create-payments.txt at line 83 [26.7446]
[26.9696]
Replacement in stack.yaml at line 5 [42.1074]
B:BD[26.10554] → [26.10554:10594]
```
    git: git@github.com:aftok/bippy.git
```
[26.10554]
[43.13697]
```
    git: https://github.com/aftok/bippy.git
```
Insertion in stack.yaml at line 15 [42.1074]
[44.23448]
```
local-bin-path: ./bin
```

Update docker build, clean up migration for payments tables.

Dependencies

In channels

Change contents

File deletion: notes.md

Insertion in .dockerignore at line 1 [8.1]

Insertion in .dockerignore at line 5 [8.1]

Insertion in .gitignore at line 19 [11.135]

Replacement in Dockerfile at line 14 [8.43]

Replacement in Dockerfile at line 19 [8.43]

Deletion in Dockerfile at line 22 [8.43]

Insertion in Dockerfile at line 31 [8.43]

Replacement in Dockerfile at line 37 [8.43]

Deletion in Dockerfile at line 44 [8.43]

Deletion in Dockerfile at line 47 [8.43]

Replacement in Makefile at line 2 [3.1254]

Replacement in Makefile at line 4 [3.1254]

File addition: bin (d--x------)

File addition: .keep (----------)

File deletion: stack.yaml

File addition: docs (d--x------)

File move: README.md → design.md

File move: features.md → features.md

File move: inception.md → inception.md

File move: MAYBE.md → maybe.md

File move: TASKS.md → tasks.md

Replacement in lib/Aftok/Database/PostgreSQL.hs at line 218 [22.456]

Replacement in lib/Aftok/Database/PostgreSQL.hs at line 438 [22.456]

Replacement in lib/Aftok/Database/PostgreSQL.hs at line 460 [22.456]

Replacement in lib/Aftok/Database/PostgreSQL.hs at line 465 [22.456]

Insertion in lib/Aftok/Database/PostgreSQL.hs at line 469 [22.456]

Insertion in lib/Aftok/Database.hs at line 14 [28.4936]

Replacement in lib/Aftok/Database.hs at line 56 [28.4936]

Replacement in lib/Aftok/Json.hs at line 9 [29.19]

Insertion in lib/Aftok/Json.hs at line 26 [29.19]

Replacement in lib/Aftok/Json.hs at line 105 [29.19]

Replacement in lib/Aftok/Json.hs at line 113 [29.19]

Replacement in lib/Aftok/Json.hs at line 118 [29.19]

Replacement in lib/Aftok/Json.hs at line 128 [29.19]

Replacement in lib/Aftok/Json.hs at line 132 [29.19]

Replacement in lib/Aftok/Json.hs at line 143 [29.19]

Replacement in lib/Aftok/Json.hs at line 163 [29.19]

Replacement in lib/Aftok/Json.hs at line 176 [29.19]

Replacement in lib/Aftok/Json.hs at line 183 [29.19]

Replacement in lib/Aftok/Json.hs at line 199 [29.19]

Replacement in lib/Aftok/Json.hs at line 211 [29.19]

Replacement in lib/Aftok/Json.hs at line 222 [29.19]

Insertion in lib/Aftok/Json.hs at line 228 [29.19]

Replacement in lib/Aftok/Json.hs at line 246 [29.19]

Insertion in lib/Aftok/Json.hs at line 250 [29.19]

Insertion in lib/Aftok/Json.hs at line 337 [29.19]

Insertion in lib/Aftok/Json.hs at line 344 [29.19]

Replacement in migrations/2016-12-31_03-45-17_create-payments.txt at line 58 [26.7446]

Replacement in migrations/2016-12-31_03-45-17_create-payments.txt at line 68 [26.7446]

Insertion in migrations/2016-12-31_03-45-17_create-payments.txt at line 78 [26.7446]

Insertion in migrations/2016-12-31_03-45-17_create-payments.txt at line 80 [26.7446]

Insertion in migrations/2016-12-31_03-45-17_create-payments.txt at line 83 [26.7446]

Replacement in stack.yaml at line 5 [42.1074]

Insertion in stack.yaml at line 15 [42.1074]