Q: Many of metrics are empty when setting up istio 1.2.1 - what did i do wrong? I’m getting started with Istio (1.2.1) on k8s v1.15.0 - and everything looks good except many of the attributes are empty or “unknown”. Using the bookinfo sample app fields like destinationName, destinationApp, destinationWorkload, sourceName, sourceWorkload, etc are empty (or set to unknown), while other metrics are set. The log lines below were taken from the telemetry container on the telemetry pod while I curl’d the URL 100.96.76.176:9080/productpage.
All the pods have app and version labels set, and I’m not sure where else to look.
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.625780Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.6.140”,“destinationName”:"",“destinationNamespace”:“g-demo”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“details.g-demo.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:"",“httpAuthority”:“details:9080”,“latency”:“2.511893ms”,“method”:“GET”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:677,“referer”:"",“reporter”:“destination”,“requestId”:“5662e4c0-8cd8-4212-b4e3-6fe8be75d3f5”,“requestSize”:0,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:178,“responseTimestamp”:“2019-07-24T16:52:24.628198Z”,“sentBytes”:313,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/details/0",“userAgent”:“curl/7.58.0”,“xForwardedFor”:“0.0.0.0”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.625458Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“unknown”,“destinationApp”:"",“destinationIp”:“100.97.6.140”,“destinationName”:"",“destinationNamespace”:"",“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“details.g-demo.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:"",“httpAuthority”:“details:9080”,“latency”:“3.060349ms”,“method”:“GET”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:342,“referer”:"",“reporter”:“source”,“requestId”:“5662e4c0-8cd8-4212-b4e3-6fe8be75d3f5”,“requestSize”:0,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:178,“responseTimestamp”:“2019-07-24T16:52:24.628457Z”,“sentBytes”:307,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:“g-demo”,“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/details/0",“userAgent”:“curl/7.58.0”,“xForwardedFor”:“0.0.0.0”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.631186Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“unknown”,“destinationApp”:"",“destinationIp”:“100.97.6.152”,“destinationName”:"",“destinationNamespace”:"",“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“reviews.g-demo.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:"",“httpAuthority”:“reviews:9080”,“latency”:“14.927949ms”,“method”:“GET”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:342,“referer”:"",“reporter”:“source”,“requestId”:“5662e4c0-8cd8-4212-b4e3-6fe8be75d3f5”,“requestSize”:0,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:375,“responseTimestamp”:“2019-07-24T16:52:24.646069Z”,“sentBytes”:549,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:“g-demo”,“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/reviews/0",“userAgent”:“curl/7.58.0”,“xForwardedFor”:“0.0.0.0”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.640811Z”,“instance”:“tcpaccesslog.logentry.istio-system”,“connectionDuration”:“0s”,“connectionEvent”:“open”,“connection_security_policy”:“unknown”,“destinationApp”:"",“destinationIp”:“100.97.5.90”,“destinationName”:"",“destinationNamespace”:"",“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“mongodb.g-demo.svc.cluster.local”,“destinationWorkload”:"",“protocol”:“tcp”,“receivedBytes”:0,“reporter”:“source”,“requestedServerName”:"",“responseFlags”:"",“sentBytes”:0,“sourceApp”:"",“sourceIp”:“100.97.8.150”,“sourceName”:"",“sourceNamespace”:“g-demo”,“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“totalReceivedBytes”:0,“totalSentBytes”:0}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.644777Z”,“instance”:“tcpaccesslog.logentry.istio-system”,“connectionDuration”:“4.38701ms”,“connectionEvent”:“close”,“connection_security_policy”:“unknown”,“destinationApp”:"",“destinationIp”:“100.97.5.90”,“destinationName”:"",“destinationNamespace”:"",“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“mongodb.g-demo.svc.cluster.local”,“destinationWorkload”:"",“protocol”:“tcp”,“receivedBytes”:351,“reporter”:“source”,“requestedServerName”:"",“responseFlags”:"",“sentBytes”:429,“sourceApp”:"",“sourceIp”:“100.97.8.150”,“sourceName”:"",“sourceNamespace”:“g-demo”,“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“totalReceivedBytes”:351,“totalSentBytes”:429}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] 2019-07-24T16:52:25.771171Z warn input set condition evaluation error: id=‘4’, error=‘lookup failed: ‘destination.service.host’’
[istio-telemetry-56d5966d6b-qj9z2 telemetry] 2019-07-24T16:52:25.771376Z warn input set condition evaluation error: id=‘4’, error=‘lookup failed: ‘destination.service.host’’
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.641665Z”,“instance”:“tcpaccesslog.logentry.istio-system”,“connectionDuration”:“0s”,“connectionEvent”:“open”,“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.5.90”,“destinationName”:"",“destinationNamespace”:“g-demo”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:"",“destinationWorkload”:"",“protocol”:“tcp”,“receivedBytes”:277,“reporter”:“destination”,“requestedServerName”:"",“responseFlags”:"",“sentBytes”:0,“sourceApp”:"",“sourceIp”:“100.97.8.150”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“totalReceivedBytes”:277,“totalSentBytes”:0}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.644179Z”,“instance”:“tcpaccesslog.logentry.istio-system”,“connectionDuration”:“3.633777ms”,“connectionEvent”:“close”,“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.5.90”,“destinationName”:"",“destinationNamespace”:“g-demo”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:"",“destinationWorkload”:"",“protocol”:“tcp”,“receivedBytes”:351,“reporter”:“destination”,“requestedServerName”:"",“responseFlags”:"",“sentBytes”:429,“sourceApp”:"",“sourceIp”:“100.97.8.150”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“totalReceivedBytes”:351,“totalSentBytes”:429}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.636470Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.8.150”,“destinationName”:"",“destinationNamespace”:“g-demo”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“ratings.g-demo.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:"",“httpAuthority”:“ratings:9080”,“latency”:“7.978662ms”,“method”:“GET”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:693,“referer”:"",“reporter”:“destination”,“requestId”:“5662e4c0-8cd8-4212-b4e3-6fe8be75d3f5”,“requestSize”:0,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:48,“responseTimestamp”:“2019-07-24T16:52:24.644258Z”,“sentBytes”:166,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/ratings/0",“userAgent”:“curl/7.58.0”,“xForwardedFor”:“0.0.0.0”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.631485Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.6.152”,“destinationName”:"",“destinationNamespace”:“g-demo”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“reviews.g-demo.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:"",“httpAuthority”:“reviews:9080”,“latency”:“14.298546ms”,“method”:“GET”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:677,“referer”:"",“reporter”:“destination”,“requestId”:“5662e4c0-8cd8-4212-b4e3-6fe8be75d3f5”,“requestSize”:0,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:375,“responseTimestamp”:“2019-07-24T16:52:24.645688Z”,“sentBytes”:555,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/reviews/0",“userAgent”:“curl/7.58.0”,“xForwardedFor”:“0.0.0.0”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.619969Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.4.119”,“destinationName”:"",“destinationNamespace”:“g-demo”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“productpage.g-demo.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:"",“httpAuthority”:“100.96.76.176:9080”,“latency”:“28.389479ms”,“method”:“GET”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:647,“referer”:"",“reporter”:“destination”,“requestId”:“5662e4c0-8cd8-4212-b4e3-6fe8be75d3f5”,“requestSize”:0,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:5179,“responseTimestamp”:“2019-07-24T16:52:24.648264Z”,“sentBytes”:5324,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/productpage",“userAgent”:“curl/7.58.0”,“xForwardedFor”:“0.0.0.0”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.619518Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“unknown”,“destinationApp”:"",“destinationIp”:“100.97.4.119”,“destinationName”:"",“destinationNamespace”:"",“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“productpage.g-demo.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:"",“httpAuthority”:“100.96.76.176:9080”,“latency”:“29.187307ms”,“method”:“GET”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:224,“referer”:"",“reporter”:“source”,“requestId”:“5662e4c0-8cd8-4212-b4e3-6fe8be75d3f5”,“requestSize”:0,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:5179,“responseTimestamp”:“2019-07-24T16:52:24.648640Z”,“sentBytes”:5318,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:“g-demo”,“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/productpage",“userAgent”:“curl/7.58.0”,“xForwardedFor”:“0.0.0.0”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] gc 74945 @116322.022s 10%: 0.015+35+105 ms clock, 0.12+3.6/8.2/36+845 ms cpu, 15->16->7 MB, 16 MB goal, 8 P
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:24.635940Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“unknown”,“destinationApp”:"",“destinationIp”:“100.97.8.150”,“destinationName”:"",“destinationNamespace”:"",“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“ratings.g-demo.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:"",“httpAuthority”:“ratings:9080”,“latency”:“8.73426ms”,“method”:“GET”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:362,“referer”:"",“reporter”:“source”,“requestId”:“5662e4c0-8cd8-4212-b4e3-6fe8be75d3f5”,“requestSize”:0,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:48,“responseTimestamp”:“2019-07-24T16:52:24.644626Z”,“sentBytes”:160,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:“g-demo”,“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/ratings/0",“userAgent”:“curl/7.58.0”,“xForwardedFor”:“0.0.0.0”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:25.628982Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.6.136”,“destinationName”:"",“destinationNamespace”:“istio-system”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“istio-telemetry.istio-system.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:“0”,“httpAuthority”:“mixer”,“latency”:“4.039738ms”,“method”:“POST”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:1152,“referer”:"",“reporter”:“destination”,“requestId”:“242c989e-1d28-447b-b382-b194574cbdd7”,“requestSize”:825,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:5,“responseTimestamp”:“2019-07-24T16:52:25.632940Z”,“sentBytes”:141,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/istio.mixer.v1.Mixer/Report",“userAgent”:"",“xForwardedFor”:“100.97.6.140”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:25.630094Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.6.136”,“destinationName”:"",“destinationNamespace”:“istio-system”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“istio-telemetry.istio-system.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:“0”,“httpAuthority”:“mixer”,“latency”:“3.013082ms”,“method”:“POST”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:1806,“referer”:"",“reporter”:“destination”,“requestId”:“219ef409-2f09-4c89-acb2-d5da0ef45169”,“requestSize”:1475,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:5,“responseTimestamp”:“2019-07-24T16:52:25.633022Z”,“sentBytes”:141,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/istio.mixer.v1.Mixer/Report",“userAgent”:"",“xForwardedFor”:“100.97.4.119”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:25.640448Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.6.136”,“destinationName”:"",“destinationNamespace”:“istio-system”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“istio-telemetry.istio-system.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:“0”,“httpAuthority”:“mixer”,“latency”:“1.354099ms”,“method”:“POST”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:964,“referer”:"",“reporter”:“destination”,“requestId”:“968c7073-d926-43f7-b8bc-3c45e273c95c”,“requestSize”:637,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:5,“responseTimestamp”:“2019-07-24T16:52:25.641721Z”,“sentBytes”:141,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/istio.mixer.v1.Mixer/Report",“userAgent”:"",“xForwardedFor”:“100.97.8.150”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:25.642777Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.6.136”,“destinationName”:"",“destinationNamespace”:“istio-system”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“istio-telemetry.istio-system.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:“0”,“httpAuthority”:“mixer”,“latency”:“129.565696ms”,“method”:“POST”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:827,“referer”:"",“reporter”:“destination”,“requestId”:“7758c475-47ea-4dab-8240-0a9bf3f08839”,“requestSize”:501,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:5,“responseTimestamp”:“2019-07-24T16:52:25.772309Z”,“sentBytes”:143,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/istio.mixer.v1.Mixer/Report",“userAgent”:"",“xForwardedFor”:“100.97.5.90”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:25.647082Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.6.136”,“destinationName”:"",“destinationNamespace”:“istio-system”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“istio-telemetry.istio-system.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:“0”,“httpAuthority”:“mixer”,“latency”:“125.353066ms”,“method”:“POST”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:1201,“referer”:"",“reporter”:“destination”,“requestId”:“344b3520-0a89-427a-95f2-dcd83eb8c302”,“requestSize”:874,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:5,“responseTimestamp”:“2019-07-24T16:52:25.772387Z”,“sentBytes”:143,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/istio.mixer.v1.Mixer/Report",“userAgent”:"",“xForwardedFor”:“100.97.6.152”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:25.648757Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.6.136”,“destinationName”:"",“destinationNamespace”:“istio-system”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“istio-telemetry.istio-system.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:“0”,“httpAuthority”:“mixer”,“latency”:“123.705739ms”,“method”:“POST”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:1180,“referer”:"",“reporter”:“destination”,“requestId”:“3830bb1e-ee4a-4747-9b85-aa8632a05f10”,“requestSize”:849,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:5,“responseTimestamp”:“2019-07-24T16:52:25.772425Z”,“sentBytes”:143,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/istio.mixer.v1.Mixer/Report",“userAgent”:"",“xForwardedFor”:“100.97.4.119”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:25.644416Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.6.136”,“destinationName”:"",“destinationNamespace”:“istio-system”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“istio-telemetry.istio-system.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:“0”,“httpAuthority”:“mixer”,“latency”:“128.10414ms”,“method”:“POST”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:1158,“referer”:"",“reporter”:“destination”,“requestId”:“7ff3a60a-b806-4f09-9c12-28dc5652a413”,“requestSize”:831,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:5,“responseTimestamp”:“2019-07-24T16:52:25.772461Z”,“sentBytes”:143,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/istio.mixer.v1.Mixer/Report",“userAgent”:"",“xForwardedFor”:“100.97.8.150”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:25.648347Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.6.136”,“destinationName”:"",“destinationNamespace”:“istio-system”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“istio-telemetry.istio-system.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:“0”,“httpAuthority”:“mixer”,“latency”:“124.187413ms”,“method”:“POST”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:1163,“referer”:"",“reporter”:“destination”,“requestId”:“678d4923-e1d8-450b-a0c4-620c89e6e088”,“requestSize”:840,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:5,“responseTimestamp”:“2019-07-24T16:52:25.772496Z”,“sentBytes”:143,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/istio.mixer.v1.Mixer/Report",“userAgent”:"",“xForwardedFor”:“100.97.6.157”}
[istio-telemetry-56d5966d6b-qj9z2 telemetry] {“level”:“info”,“time”:“2019-07-24T16:52:25.644908Z”,“instance”:“accesslog.logentry.istio-system”,“apiClaims”:"",“apiKey”:"",“clientTraceId”:"",“connection_security_policy”:“none”,“destinationApp”:"",“destinationIp”:“100.97.6.136”,“destinationName”:"",“destinationNamespace”:“istio-system”,“destinationOwner”:"",“destinationPrincipal”:"",“destinationServiceHost”:“istio-telemetry.istio-system.svc.cluster.local”,“destinationWorkload”:"",“grpcMessage”:"",“grpcStatus”:“0”,“httpAuthority”:“mixer”,“latency”:“221.122459ms”,“method”:“POST”,“permissiveResponseCode”:“none”,“permissiveResponsePolicyID”:“none”,“protocol”:“http”,“receivedBytes”:1188,“referer”:"",“reporter”:“destination”,“requestId”:“780ce5c5-df66-47a8-abcf-349ff7a9cf3e”,“requestSize”:861,“requestedServerName”:"",“responseCode”:200,“responseFlags”:"-",“responseSize”:5,“responseTimestamp”:“2019-07-24T16:52:25.866003Z”,“sentBytes”:143,“sourceApp”:"",“sourceIp”:“0.0.0.0”,“sourceName”:"",“sourceNamespace”:"",“sourceOwner”:"",“sourcePrincipal”:"",“sourceWorkload”:"",“url”:"/istio.mixer.v1.Mixer/Report",“userAgent”:"",“xForwardedFor”:“100.97.6.152”}

Thank you,
Mike

A: I am posting @ShreeGrossOptum's answer from the comments for better visibility and due to lack of response from the OP:

Sir please check once about attributes I hope you ll get your
  answers like why mostly attributes are unknown and empty.


Q: htaccess mod_rewrite good in IE, but downloads web page in other browsers (Chrome, Firefox) I am trying to modify an .htaccess file so that file extensions are omitted-- mainly .html  It seems to be working fine in IE written like this:

RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME}.html -f
RewriteRule ^(.*)$ $1.html
AddType text/html .html

But in Chrome or Firefox (haven't tried any others) the web page immediately downloads instead of displays without the extension.
I have never modified this file before-- I got the above Rewrite lines from another site but I cannot seem to find any solution to prevent the files from downloading in those browsers. This site is hosted on Amazon Web Services.
Thanks


Q: Play Console Unable To Click Create New Release Button My Application is published in the play store but I want to update my app but I am unable to update my app in Google Play Console


Q: Change Image Button in Adapter Listview Can Anyone help me. Where is my fault?
I have an adapter listview is:
db = Database.getReadableDatabase();
            String queryString = "SELECT * from TBL_CHECKIN WHERE FLAG='1' and NOCUST='" + ListViewCallplan.get(position).gettxid() + "'";
            Cursor c = db.rawQuery(queryString, null);
            if (c.getCount() == 0) {
                System.out.println("ok1");
                notifyDataSetInvalidated();
                holder.image .setImageResource(R.mipmap.cekin);
                holder.image.setOnClickListener(new View.OnClickListener() {
  @Override
                        public void onClick(View v) {
                            Intent logtent = new Intent(context, Cls_Checkin.class);
                            logtent.putExtra("custcode", ListViewCallplan.get(position).gettxid());
                            context.startActivity(logtent);
                        }
                    });
    //                notifyDataSetChanged();
                }
                if (c.getCount() > 0) {
                    System.out.println("okcekot");
                    notifyDataSetInvalidated();
                    holder.image .setImageResource(R.mipmap.cekout);
                    holder.image.setOnClickListener(new View.OnClickListener() {
                        @Override
                        public void onClick(View v) {
                            Intent logtent = new Intent(context, Cls_Checkout.class);
                            context.startActivity(logtent);
                        }
                    });
    //                notifyDataSetChanged();
                }

I want to change image the button with that condition, but when me open again it's not do nothing. Stay in same display. The image in button always show checkin for the example.


Q: SharePoint Approval I need to add sharepoint approval to a list... and I was hoping to use the default approval process... but...  I need to run some code once an item is approved.  Do I have to then use VS to create a custom workflow?
I need to run some code that currently runs in an event reciever.  I need to move the code to another function (as we are moving the processing out of event reciever code and now based on item approval).  This code would execute after the item is approved and can be hosted as a service or .net code.

A: Full commented source to a state-machine based approval workflow comes with the MOSS SDK 1.5 in the samples directory.
http://www.microsoft.com/downloads/details.aspx?familyid=6D94E307-67D9-41AC-B2D6-0074D6286FA9
-Oisin


Q: Unable to cast object of type 'OpenQA.Selenium.Chrome.ChromeDriver' to type 'OpenQA.Selenium.Remote.RemoteWebDriver' I am trying to use WebDriverManager Package for .Net for automatically managing the chromedriver.exe version while executing a test case. But when ever I try to type cast ChromeDriver object to RemoteDriverObject it is showing me below error.
Unable to cast object of type 'OpenQA.Selenium.Chrome.ChromeDriver' to type 'OpenQA.Selenium.Remote.RemoteWebDriver'.

I will write the methods and the code that I am using below:
public IWebDriver WebInit()
{
     ChromeOptions options = new ChromeOptions();
     options.AddArguments("--test-type");
     options.AddArguments("--start-maximized");
     options.AddArguments("--disable-infobars");
     options.AddArguments("--disable-extensions");
     options.AddAdditionalCapability("useAutomationExtension", false);
     options.AddArguments("--ignore-certificate-errors");
     options.AddExcludedArgument("enable-automation");
     options.AddUserProfilePreference("credentials_enable_service", false);
     options.AddUserProfilePreference("profile.password_manager_enabled", false);
     options.AddAdditionalCapability("useAutomationExtension", false);
     options.AddUserProfilePreference("download.prompt_for_download", false);
     options.AddUserProfilePreference("download.default_directory", downloadFilepath);
     options.AddUserProfilePreference("safebrowsing.enabled", true);
     options.AddArguments("window-size=1600,900");
     new DriverManager().SetUpDriver(new ChromeConfig(), VersionResolveStrategy.MatchingBrowser); //this is the code used from the mentioned Github Repository
     _driver = new ChromeDriver(options);
}

public void FunctionWhereIAmCallingTheWebInitFucntion()
{
     _driver = WebInit()
     ICapabilities capabilities = ((RemoteWebDriver)_driver).Capabilities; //whenever this line gets executed it throws the exception that is mentioned
}

Below are the Package versions that I am using
<PackageReference Include="Selenium.Support" Version="4.1.1" />
<PackageReference Include="Selenium.WebDriver" Version="4.1.1" />
<PackageReference Include="WebDriverManager" Version="2.12.4" />

Please can anyone from the community can guide me where am I making a mistake? Thank you very much!!

A: Ok, based on your response in the comments, I think this is what you are going for.
The ChromeDriver object has a Capabilities property you can use to request the name and version.
As long as you are working with a ChromeDriver directly and not an IWebDriver that property is accessible like follows:
string? versions = driver.Capabilities.GetCapability("browserVersion").ToString();
string? name = driver.Capabilities.GetCapability("browserName").ToString();

I modified your program a little bit based on what it seems like you are trying to do and printed the result to the console:
private static ChromeDriver WebInit()
{
    // set up options
    ChromeOptions options = new ChromeOptions();
    options.AddArguments("--start-maximized");

    // download latest chromedriver
    new DriverManager().SetUpDriver(new ChromeConfig(), VersionResolveStrategy.MatchingBrowser); //this is the code used from the mentioned Github Repository

    // initialize the driver
    ChromeDriver driver = new ChromeDriver(options);

    return driver;
}

public static void Main()
{
    ChromeDriver driver = WebInit();

    // get the capabilities
    string? versions = driver.Capabilities.GetCapability("browserVersion").ToString();
    string? name = driver.Capabilities.GetCapability("browserName").ToString();

    Console.WriteLine(versions);
    Console.WriteLine(name);

    Console.ReadKey();
}

The above example works for me in .Net 6.0 with the following NuGet packages installed:

*

*Selenium.Support (4.1.1)

*Selenium.WebDriver (4.1.1)

*WebDriverManager (2.13.0)


Q: move up and down a treeview I'm populating a tkinter treeview in python using Os.walk() but after populating it I want to be able to reorder the tree leaves using buttons.
The command for moving up the tree works just fine (I want to be able to move multiple leaves at once)
def moveUp():
    leaves = Tree.selection()
    for i in leaves:
        Tree.move(i, Tree.parent(i), Tree.index(i)-1)

But when I reversed it to go down the tree I get a weird bug
def moveDown():
    leaves = Tree.selection()
    for i in leaves:
        Tree.move(i, Tree.parent(i), Tree.index(i)+1)

I can only move a single leaf down, if I select an odd number of leaves then the lowest one moves down, and if I pick an even number of leaves none of them move.

A: As suggested in the comments going through leaves in the reverse order using reversed() fixes the problem. (Would be a comment but I don't have the reputation)
def moveDown():
    leaves = Tree.selection()
    for i in reversed(leaves):
        Tree.move(i, Tree.parent(i), Tree.index(i)+1)

A: Note, for moveUp, reversing the leaves would result in nothing actually changing in the tree. Simply iterating through the leaves in order is the ticket.
def moveUp():
    leaves = Tree.selection()
    for i in leaves:
        Tree.move(i, Tree.parent(i), Tree.index(i)-1)


Q: Slow SQL: Select count of Table1 values missing from Table2 I am trying to return a count of the number of values from Table1 WHERE NOT EXISTS in Table2, I've got a query that seems correct but it's extremely slow to run:
SELECT count(container_no)
FROM pier_data
WHERE NOT EXISTS(
    SELECT 1
    FROM Iron_mountain_data
    WHERE pier_data.container_no = iron_mountain_data.[Customer Box Nbr]
    );

Is there any way I can speed this up?
Edit: For whatever reason I believe this LEFT JOIN business gave me the most expedient results in MS Access:
SELECT Count(container_no) AS boxes_missing_from_IM
FROM pier_data AS pd LEFT JOIN iron_mountain_data AS imd ON pd.container_no =
imd.[Customer Box Nbr]
WHERE imd.[Customer Box Nbr] Is Null;

A: This might be better, try it :
SELECT COUNT(container_no) FROM pier_date
WHERE container_no NOT IN
(
     SELECT [Customer Box Nbr] FROM iron_mountain_data
);

Also, as it has been suggested, you could use a left join with a where clause like this :
SELECT COUNT(container_no) FROM pier_date pd
LEFT JOIN iron_mountain_data imd ON pd.container_no = imd.[Customer Box Nbr]
WHERE imd.[Customer Box Nbr] IS NULL

The use of keyword 'DISTINCT' might be useful too like
COUNT(DISTINCT container_no)

or
SELECT DISTINCT [Customer Box Nbr]

A: Assuming [container_no] and [Customer box nbr] captures the same information
    SELECT  COUNT(container_no)
    FROM pier_date pd
    WHERE   container_no NOT IN (
                                    SELECT DISTINCT [Customer Box Nbr]
                                    FROM            iron_mountain_data
                                )

A: Your query is fine, although I would use COUNT(*) and table aliases:
SELECT COUNT(*)
FROM pier_data as pd
WHERE NOT EXISTS (SELECT 1
                  FROM Iron_mountain_data as imd
                  WHERE pd.container_no = imd.[Customer Box Nbr]
                 );

(These changes have no impact on performance, except for a very, very, very minor check that container_no is not NULL for the COUNT().)
For performance, you want an index on Iron_mountain_data([Customer Box Nbr]):
create index idx_iron_mount_data_customer_box_nbr on Iron_mountain_data([Customer Box Nbr];

This should work for almost any way that you write the query.


Q: How to test the appearance of a child component within an ngIf Statement in Angular 8 I have a parent component where one of two child components are displayed, depending on the value of a variable (workHistoryState.ViewType):

        <app-job-list></app-job-list>

        <app-assignment-list ></app-assignment-list>

The value in workHistoryState.ViewType is updated by another child component. It calls this method on the workHistoryState service:
public ViewTypeChanged(viewType: string) {
    this.ViewType = viewType;
}

In my test code for the parent component I have to use mocks for the child components. The mocks are as follows:
import { Component } from "@angular/core";
@Component({
    selector: 'app-assignment-list',
    template: '
Mock Assignment List Component
'
})
export class AssignmentListComponentMock {}

import { Component } from "@angular/core";
@Component({
    selector: 'app-job-list',
    template: '
Mock Job List Component
'
})
export class JobListComponentMock {}

So my test code is as follows (and both of the tests fail):
import { async, ComponentFixture, TestBed } from '@angular/core/testing';
import { WorkHistoryParentComponent } from './work-history-parent.component';
import { JobListComponentMock } from '../component-mocks/job-list.component.mock';
import { AssignmentListComponentMock } from '../component-mocks/assignment-list.component.mock';
import { IndustryDropDownComponentMock } from '../component-mocks/industry-drop-down.component.mock';
import { RoleTypeDropDownComponentMock } from '../component-mocks/role-type-drop-down.component.mock';
import { ViewTypeSelectComponentMock } from '../component-mocks/view-type-select.component.mock';
import { By } from '@angular/platform-browser';
import { APP_BASE_HREF } from '@angular/common';

describe('WorkHistoryParentComponent', () => {
    let component: WorkHistoryParentComponent;
    let fixture: ComponentFixture<WorkHistoryParentComponent>;
beforeEach(async(() => {
  TestBed.configureTestingModule({
  declarations: [WorkHistoryParentComponent, JobListComponentMock, AssignmentListComponentMock, IndustryDropDownComponentMock, RoleTypeDropDownComponentMock, ViewTypeSelectComponentMock],
  providers:[
    { provide: APP_BASE_HREF, useValue : '/' },

],
}).compileComponents();
}));

beforeEach(() => {
  fixture = TestBed.createComponent(WorkHistoryParentComponent);
  component = fixture.componentInstance;
  fixture.detectChanges();
});

it('should not find job list by css when detailed view is chosen', () => {
    component.workHistoryState.ViewTypeChanged("detailed");

    let jobList = fixture.debugElement.query(By.css('.jobList'));

    expect(jobList).toBeNull();
});

it('should not find job list when detailed view is chosen', () => {
    component.workHistoryState.ViewTypeChanged("detailed");

    let jobList = fixture.debugElement.query(By.directive(JobListComponentMock));

    expect(jobList).toBeNull();
    });
});

Does anybody know why they are failing?

A: I needed to call fixture.detectChanges() after changing the value in the service. So the tests should have been:
it('should not find job list by css when detailed view is chosen', () => {
  component.workHistoryState.ViewTypeChanged("detailed");
  fixture.detectChanges();

  let jobList = fixture.debugElement.query(By.css('.jobList'));

  expect(jobList).toBeNull();
});

it('should not find job list when detailed view is chosen', () => {
  component.workHistoryState.ViewTypeChanged("detailed");
  fixture.detectChanges();

  let jobList = fixture.debugElement.query(By.directive(JobListComponentMock));

  expect(jobList).toBeNull();
});


Q: I'm trying to add a List-Unsubscribe header to my e-mail header using php-mailer I want to show unsubscribe links beside the email headers of email providers like Gmail, yahoo, and outlook.
This is how its called in PHP-mailer:(I have tried by following ways)
1st way:
$mail->AddCustomHeader("List-Unsubscribe:<mailto:example@yopmail.com>,<https://myexample.com/unsubscribe.php?identifier=1599&type=s&campaignid=99>");

2nd way:
$mail->AddCustomHeader("List-Unsubscribe:<mailto:example@yopmail.com?subject=unsubscribe>,<https://myexample.com/unsubscribe.php?identifier=1599&type=s&campaignid=99>");

By this code, it sends the header List-Unsubscribe with the email but the unsubscribe link is not showing there. I don't know this is the process or not.
I have DKIM: PASS record.
In the mail raw header of Gmail, outlook, and yahoo it shows, List-Unsubscribe: List-Unsubscribe: mailto:example@yopmail.com?subject=unsubscribe,https://myexample.com/unsubscribe.php?identifier=1599&type=s&campaignid=99. Do we need to do some email server configuration for this to show link in emails header ? I can not find the solution for this.


Q: Run Celery tasks on Railway I deployed a Django project in Railway, and it uses Celery and Redis to perform an scheduled task. The project is successfully online, but the Celery tasks are not performed.
If I execute the Celery worker from my computer's terminal using the Railway CLI, the tasks are performed as expected, and the results are saved in the Railway's PostgreSQL, and thus those results are displayed in the on-line site. Also, the redis server used is also the one from Railway.
However, Celery is operating in 'local'. This is the log on my local terminal showing the Celery is running local, and the Redis server is the one up in Railway:
-------------- celery@MacBook-Pro-de-Corey.local v5.2.7 (dawn-chorus)
--- ***** -----
-- ******* ---- macOS-13.1-arm64-arm-64bit 2023-01-11 23:08:34
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:  suii:0x1027e86a0
- ** ---------- .> transport:  redis://default:**@containers-us-west-28.railway.app:7078//
- ** ---------- .> results:
- *** --- * --- .> concurrency: 10 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery  exchange=celery(direct) key=celery
[tasks]
. kansoku.tasks.suii_kakunin

I included this line of code in the Procfile regarding to the worker (as I saw in another related answer):
worker: python manage.py qcluster --settings=my_app_name.settings

And also have as an environment variable CELERY_BROKER_REDIS_URL pointing to Railway's REDIS_URL. I also tried creating a 'Periodic task' from the admin of the live aplication, but it just doesn't get executed. What should I do in order to have the scheduled tasks be done automatically without my PC?


Q: while changing SDK 33 XML autocompletion is not working I hv version bubble bee Android after changing SDK 33 XML auto complete is not working plz help. when I change version 33 then Android application execute but XML auto complete doesn't work
I have tried all possible ways but i unable to find and solution


Q: AJAX POST JSON Array Javascript NOT JQUERY
In this screenshot, a JSON array is being passed to PHP via AJAX without a POST body. I am using this code to implement it:
with(x=new XMLHttpRequest()) open("POST", "http://myweb/api/mobile/v1/jobeventadd?key=cfff"), setRequestHeader("Content-Type", "application/x-www-form-urlencoded"), send("%7B%0A%22SessionID%22%3A%22hn0oqa0u687avsrnev6f5t2nh7%22%2C%0A%22ObjectID%22%3A%226460%22%2C%0A%22ItemName%22%3A%22UologiciPhone%20test%20event%22%2C%0A%22ActivityFrom%22%3A%2201-01-2013%2012%3A00%3A00%22%0A%7D");

But it's not working. How can I do it via XMLHttpRequest? No JQUERY PLEASE

A: Use the onreadystatechange property to get the response after the success state, and then store the data in a custom header to troubleshoot issues with the POST body:
with(new XMLHttpRequest)
  {
  open("POST",{},true);
  setRequestHeader("Foo", "Bar");
  send("");
  onreadystatechange = handler;
  }

function handler(event)
 {
 !!event.target && !!event.target.readyState && event.target.readyState === 4 && ( console.log(event) );
 }

References

*

*XMLHTTPRequest Living Standard

*Fetch Living Standard

A: It was same as GET
with(new XMLHttpRequest)
  {
  open("POST","http://google.com",true);
  send("hello=world&no=yes");
  onreadystatechange = function(){};
  }


Q: django: generic views + custom template tags or custom views + generic/normal template tags This is more of a best-practices question, and given that I'm quite tired it mightn't make much sense.
I've been putting together a blog app as a learning experience and as an actual part of a website I am developing.
I've designed it like most apps so that you can list blog posts by multiple criteria i.e.
/blog/categories/
/blog/authors/
/blog/tags/
/blog/popular/
etc.

On each page above I also want to list how many entries are a part of that criteria
i.e. for "categories", I want /blog/categories/ to list all the different categories, but also mention how many blog posts are in that category, and possibly list those entries.
Django seems to give you lots of ways of doing this, but not much indication on what's best in terms of flexibility, reusability and security.
I've noticed that you can either
A: Use generic/very light views, pass a queryset to the template, and gather any remaining necessary information using custom template tags.
i.e. pass the queryset containing the categories, and for each category use a template tag to fetch the entries for that category
or B: Use custom/heavy views, pass one or more querysets + extra necessary information through the view, and use less template tags to fetch information.
i.e. pass a list of dictionaries that contains the categories + their entries.
The way I see it is that the view is there to take in HTTP requests, gather the required information (specific to what's been requested) and pass the HTTP request and Context to be rendered. Template tags should be used to fetch superflous information that isn't particularly related to the current template, (i.e. get the latest entries in a blog, or the most popular entries, but they can really do whatever you like.)
This lack of definition (or ignorance on my part) is starting to get to me, and I'd like to be consistent in my design and implementation, so any input is welcome!

A: I'd say that your understanding is quite right. The main method of gathering information and rendering it via a template is always the view. Template tags are there for any extra information and processing you might need to do, perhaps across multiple views, that is not directly related to the view you're rendering.
You shouldn't worry about making your views generic. That's what the built-in generic views are for, after all. Once you need to start stepping outside what they provide, then you should definitely make them specific to your use cases. You might of course find some common functionality that is used in multiple views, in which case you can factor that out into a separate function or even a context processor, but on the whole a view is a standalone bit of code for a particular specific use.


Q: How do I set up a Jenkins CI server to run automated BDD selenium tests using a remote webdriver? I'm new to Jenkins and am trying to set up a server to run selenium tests from a GitHub repo. I'm sure that I'm doing something wrong, likely several things, but haven't been able to figure it out.
I have configured the selenium plugin to use the default Selenium hub port 4444.
Project GitHub Configuration
Can't figure out why I'm getting this error. The credentials match the created username and ssh key. I can even access the repo by clicking on GitHub in the project dashboard.
Project Shell Execution Steps
The before-build execution steps. These are the commands I use in the terminal to run the tests locally.
When I build the job it gives the following log:
 Started by <user>
 Building in workspace /Users/Shared/Jenkins/Home/workspace/Tutorial
 > git rev-parse --is-inside-work-tree # timeout=10
 Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/<repo address>.git # timeout=10
 Fetching upstream changes from https://github.com/<repo address>.git
 > git --version # timeout=10
 using GIT_SSH to set credentials
 > git fetch --tags --progress https://github.com/<repo address>.git +refs/heads/*:refs/remotes/origin/*
 ERROR: Error fetching remote repo 'origin'
 hudson.plugins.git.GitException: Failed to fetch from https://github.com/<repo address>.git
 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:809)
 at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1076)
 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1107)
 at hudson.scm.SCM.checkout(SCM.java:496)
 at hudson.model.AbstractProject.checkout(AbstractProject.java:1281)
 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604)
 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
 at hudson.model.Run.execute(Run.java:1728)
 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
 at hudson.model.ResourceController.execute(ResourceController.java:98)
 at hudson.model.Executor.run(Executor.java:405)
 Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress https://github.com/<repo address>.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
 stdout:
 stderr: remote: Invalid username or password.
 fatal: Authentication failed for 'https://github.com/<repo address>.git/'

 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1877)
 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1596)
 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:71)
 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:348)
 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:807)
 ... 11 more
 ERROR: null
 Finished: FAILURE

A: If you are using SSH key for Jenkins to authenticate try using SSH version e.g. git@github.com:FOO/BAR.git instead of HTTPS one.


Q: Spark Session Error: java.lang.IllegalArgumentException: requirement failed: Can only call getServletHandlers on a running MetricsSystem I am unable to build a SparkSession in Scala without an error. I am using Maven as my build tool. I have Spark 3.3.0 installed locally. I also have an Azure Databricks 3.3.0 cluster. Whenever I pass the databricks master URL it bombs out with this error:
Error: java.lang.IllegalArgumentException: requirement failed: Can only call getServletHandlers on a running MetricsSystem
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>sample</groupId>
    <artifactId>sample</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <spark.version>3.3.0</spark.version>
        <scala.version.major>2.12</scala.version.major>
        <scala.version.minor>12</scala.version.minor>
        <scala.version>${scala.version.major}.${scala.version.minor}</scala.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>${scala.version}</version>
        </dependency>

        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-mllib_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
        </dependency>
        <dependency>
            <groupId>org.jmockit</groupId>
            <artifactId>jmockit</artifactId>
            <version>1.34</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>net.alchim31.maven</groupId>
                <artifactId>scala-maven-plugin</artifactId>
                <version>3.2.2</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <scalaVersion>${scala.version}</scalaVersion>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.3</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

package sample

import org.apache.spark.SparkConf
import org.apache.spark.sql.SparkSession

object SparkMain extends App {

  val sparkConf = new SparkConf()
    .setMaster("local[1]")
    .setAppName("ScalaSparkDF-Test")
    .set("spark.driver.allowMultipleContexts", "false")
    .set("spark.ui.enabled", "false")

  val spark = SparkSession.builder()
    .config(sparkConf)
    .getOrCreate()

  val df = spark.read.option("header","true")
    .csv("/Users/")

}


Q: Angular: get request i have an issue with displaying data in console. I am trying to retrieve log data in the console but i am getting null without knowing what the problem is. This is my code:
Service.ts
 getItemById(id:number): Observable<any> {
    return this.http.get<any>(`${this.API}/${id}`).pipe(catchError(this.handleError
    ));
  }

  private handleError(httpErrorResponse:HttpErrorResponse){
    if(httpErrorResponse.error instanceof ErrorEvent){
      console.error(httpErrorResponse.error.message)
    }
    return of(null)
  }

Component.ts
showItem(id: any) {

    this.ItemService.getItemById(id).subscribe((data: any) => {
      console.log(data)

      this.log = data;

    })
  }

html button
        <button class="btn btn-block btn-primary" (click)="showItem(log.id)">{{buttonName}}</button>

when i click the button the response is giving 200 in the network section

but the console is returning a null, it does not return the payload.

Any solutions ?

A: Since you have an error, try first with a simpler solution:
Service.ts
getItemById(id:number): Observable<any> {
   return this.http.get(`${this.API}/${id}`);
}

Component.ts
showItem(id: any) {

   this.ItemService.getItemById(id)
   .subscribe(
      (data: any) => {
         console.log(data);
         //this.log = data;
    });

}

After this, see if your API works (if it gets data).
Once you check this,

*

*If it doesn't return anything, your solution was right, and the problem is in the back.

*If it is returning data, there was a problem. Then, in order to find out what is going on, you can try to left this simplified version and try the error directly in the subscription, with catchError or simply managing the "path" of error (something like this:)
this.ItemService.getItemById(id)
   .subscribe(
      (data: any) => {
         console.log(data);
         //this.log = data;
      },
      err => {console.log(err);} // You'll get the whole object,
                                 // If you get [object object] you shoud use the specifics fields, something like console.log(err.error.message) or something like that... (look them up on the internet)
    );

A: I just figured out that i had to add the responseType to the url in the service.ts.
This link was useful :   Angular HttpClient "Http failure during parsing"


Q: Elastic Search missing some documents while creating index from another index I have created one index in elastic search name as documents_local and load data from Oracle Database via logstash. This index containing following values.
"FilePath" : "Path of the file",
"FileName" : "filename.pdf",
"Language" : "Language_name"

Then, i want to index those file contents also, so i have created one more index in elastic search named as document_attachment and using the following Java code, i fetched FilePath & FileName from the index documents_local with the help of the filepath i retrived file will is available in my local drive and i have index those file contents using ingest-attachment plugin processor.
Please find my java code below, where am indexing the files.
private final static String INDEX = "documents_local";  //Documents Table with file Path - Source Index
private final static String ATTACHMENT = "document_attachment"; // Documents with Attachment...  -- Destination Index
private final static String TYPE = "doc";

public static void main(String args[]) throws IOException {

    RestHighLevelClient restHighLevelClient = null;
    Document doc=new Document();

    try {
        restHighLevelClient = new RestHighLevelClient(RestClient.builder(new HttpHost("localhost", 9200, "http"),
                new HttpHost("localhost", 9201, "http")));
    } catch (Exception e) {
        System.out.println(e.getMessage());
    }

    //Fetching Id, FilePath & FileName from Document Index.
    SearchRequest searchRequest = new SearchRequest(INDEX);
    searchRequest.types(TYPE);
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    QueryBuilder qb = QueryBuilders.matchAllQuery();
    searchSourceBuilder.query(qb);
    searchSourceBuilder.size(3000);
    searchRequest.source(searchSourceBuilder);
    SearchResponse searchResponse = null;
    try {
         searchResponse = restHighLevelClient.search(searchRequest);
    } catch (IOException e) {
        e.getLocalizedMessage();
    }

    SearchHit[] searchHits = searchResponse.getHits().getHits();
    long totalHits=searchResponse.getHits().totalHits;

    int line=1;
    String docpath = null;

    Map<String, Object> jsonMap ;
    for (SearchHit hit : searchHits) {

        String encodedfile = null;
        File file=null;

        Map<String, Object> sourceAsMap = hit.getSourceAsMap();
        doc.setId((int) sourceAsMap.get("id"));
        doc.setLanguage(sourceAsMap.get("language"));
        doc.setFilename(sourceAsMap.get("filename").toString());
        doc.setPath(sourceAsMap.get("path").toString());

        String filepath=doc.getPath().concat(doc.getFilename());

        System.out.println("Line Number--> "+line+++"ID---> "+doc.getId()+"File Path --->"+filepath);

        file = new File(filepath);
        if(file.exists() && !file.isDirectory()) {
            try {
                FileInputStream fileInputStreamReader = new FileInputStream(file);
                byte[] bytes = new byte[(int) file.length()];
                fileInputStreamReader.read(bytes);
                encodedfile = new String(Base64.getEncoder().encodeToString(bytes));
            } catch (FileNotFoundException e) {
                e.printStackTrace();
            }
        }

        jsonMap = new HashMap<>();
        jsonMap.put("id", doc.getId());
        jsonMap.put("language", doc.getLanguage());
        jsonMap.put("filename", doc.getFilename());
        jsonMap.put("path", doc.getPath());
        jsonMap.put("fileContent", encodedfile);

        String id=Long.toString(doc.getId());

        IndexRequest request = new IndexRequest(ATTACHMENT, "doc", id )
                .source(jsonMap)
                .setPipeline(ATTACHMENT);

        try {
            IndexResponse response = restHighLevelClient.index(request);
        } catch(ElasticsearchException e) {
            if (e.status() == RestStatus.CONFLICT) {
            }
            e.printStackTrace();
        }

    }

    System.out.println("Indexing done...");
}

Please find my mappings details for ingest attachment plugin( i did mappping first and then executing this java code ).
    PUT _ingest/pipeline/document_attachment
    {
      "description" : "Extract attachment information",
      "processors" : [
        {
          "attachment" : {
            "field" : "fileContent"
          }
        }
      ]
    }

But while doing this process, Am missing some of the doucments,
My source index is documents_local which is having 2910 documents.
Am fetching all the 118 documents and attaching my PDF  ( after converting to base64) and writting in another index document_attachment
But document_attachment index having only 118 it should be 2910. some of the doucments are missing. Also, for indexing its taking very long time.
Am not sure, how the documents are missing in second index ( document_attachment ) and is there any other way to improvise this process ?
Can we include Thread mechanism here. Because, i future i have to index more than 100k pdf, in this same way.


Q: Purge hidden storage on Github Desktop OSX? I have several folders of jpg files on my OSX laptop, cloned from a GitHub repository. The files I can see total less than 20GB, but my system tells me that they occupy 65+GB.

Where are these hidden files? How can I purge them?
I should mention that there is only one branch in the repository.
(Also, I know that GitHub is not meant to be a place to store lots of image files. This repo is a component of a larger project.)

A: As mentioned in "Show system files / Show git ignore in osx", use in your Finder ⌘⇧.
That should show you the hidden folder .git/, which include all the history of the repo.
And for a repo of binaries (like jpeg images), and new version of an image might take quite a lot of place (poor diff, poor compression between each version)

I wonder if there's any way I can purge this history from local storage? (I'd like to keep it in the cloud)

You would need to download only the archive of the repo (meaning the content, without the history).
If you don't need to change and push back those modification, the archive would be enough.
If you need the history, a shallow clone (git clone --depth 1) that I detailed here (or here) is best to keep the downloaded data at a minimum.

A: I would try re-indexing your hdd and see if that accurately displays file sizes once its done.
Run the following in osx terminal:
    sudo mdutil -E /

Original article:
https://www.maketecheasier.com/fix-wrong-hard-drive-data-usage-calculation-osx/


Q: Auto Generate Patient ID Using Two Columns In the table Screening i have StudyID. The StudyID is a combination of two numbers StudyType( For Cases = KNH-01, For Controls = KNH-02) and Serealized 5 digit number (00001,00002). So the screening table looks like this;
ID  StudyID        Age
 1   KNH-01/00001   22
 2   KNH-01/00002   15
 3   KNH-02/00001   4
 4   KNH-02/00002   28
Currently the StudyID numbers are manually saved in a log book and I would wish to automate the process of generating the StudyID numbers. I have created a table to manage StudyID
CREATE TABLE [dbo].[PatientID](
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [PatientType] [varchar](5) NULL,
    [PatientNumber] [int] NULL,
 CONSTRAINT [PK__PatientI__3214EC2739E294A9] PRIMARY KEY CLUSTERED
(
    [ID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

GO

I am unable to get last PatientNumber  based on the PatientType my intention is to have a button at the vb.net front end when a user clicks to generate StudyID it checks the last given number within a studytype adds 1 to it and loads to form. e.g. KNH-01/00003 or KNH-02/00003. If you have a better Idea i will really appreciate.

A: Well, my personal favorite way of solving such requirements involves using a computed column and a UDF to compute the values in that column.
So first, you create the UDF:
CREATE FUNCTION dbo.GenerateStudyID (
    @PatientType varchar(5),
    @id int
)
RETURNS char(11)
AS
BEGIN
    RETURN (
        SELECT @PatientType +'/'+ RIGHT('00000', CAST(COUNT(*) as char(5)), 5)
        FROM [dbo].[PatientID]
        WHERE PatientType = @PatientType
        AND Id <= @id
    )
END
GO

Then, you add the computed column to your table:
ALTER TABLE [dbo].[PatientID]
ADD StudyID as dbo.GenerateStudyID (PatientType, id)
GO

Please note that you can't use this method if you need this column to be persisted, since once you will delete a record the values will change.


Q: uploaded files - database vs filesystem, when using Grails and MySQL I know this is something of a "classic question", but does the mysql/grails (deployed on Tomcat) put a new spin on considering how to approach storage of user's uploaded files.
I like using the database for everything (simpler architecture, scaling is just scaling the database).  But using the filesystem means we don't lard up mysql with binary files.  Some might also argue that apache (httpd) is faster than Tomcat for serving up binary files, although I've seen numbers that actually show just putting Tomcat on the front of your site can be faster than using an apache (httpd) proxy.
How should I choose where to place user's uploaded files?
Thanks for your consideration, time and thought.

A: I don't know if one can make general observations about this kind of decision, since it's really down to what you are trying to do and how high up the priority list NFRs like  performance and response time are to your application.
If you have lots of users, uploading lots of binary files, with a system serving large numbers of those uploaded binary files then you have a situation where the costs of storing files in the database include:

*

*Large size binary files

*Costly queries

Benefits are

*

*Atomic commits

*Scaling comes with database (though w MySQL there are some issues w multinode etc)

*Less fiddly and complicated code to manage file systems etc

Given the same user situation where you store to the filesystem you will need to address

*

*Scaling

*File name management (user uploads same name file twice etc)

*Creating corresponding records in DB to map to the files on disk (and the code surrounding all that)

*Looking after your apache configs so they serve from the filesystem

We had a similar problem to solve as this for our Grails site where the content editors are uploading hundreds of pictures a day. We knew that driving all that demand through the application when it could be better used doing other processing was wasteful (given that the expected demand for pages was going to be in the millions per week we definitely didn't want images to cripple us).
We ended up creating upload -> file system solution. For each uploaded file a DB meta-data record was created and managed in tandem with the upload process (and conversely read that record when generating the GSP content link to the image). We served requests off disk through Apache directly based on the link requested by the browser. But, and there is always a but, remember that with things like filesystems you only have content per machine.
We had the headache of making sure images got re-synchronised onto every server, since unlike a DB which sits behind the cluster and enables the cluster behave uniformly, files are bound to physical locations on a server.
Another problem you might run up against with filesystems is folder content size. When you start having folders where there are literally tens of thousands of files in them, the folder scan at the OS level starts to really drag. To avert this problem we had to write code which managed image uploads into yyyy/MM/dd/image.name.jpg folder structures, so that no one folder accumulated hundreds of thousands of images.
What I'm implying is that while we got the performance we wanted by not using the DB for BLOB storage, that comes at the cost of development overhead and systems management.

A: Just as an additional suggestion: JCR (eg. Jackrabbit) - a Java Content Repository. It has several benefits when you deal with a lot of binary content. The Grails plugin isn't stable yet, but you can use Jackrabbit with the plain API.

A: Another thing to keep in mind is that if your site ever grows beyond one application server, you need to access the same files from all app servers.  Now all app servers have access to the database, either because that's a single server or because you have a cluster.  Now if you store things in the file system, you have to share that, too - maybe NFS.

A: Even if you upload file in filesystem, all the files get same permission, so any logged in user can access any other's file just entering the url (Since all of them get same permission). If you however plan to give each user a directory then a user permission of apache (that is what server has permission) is given to them. You should su to root, create a user and upload files to those directories. Again accessing those files could end up adding user's group to server group. If I choose to use filesystem to store binary files, is there an easier solution than this, how do you manage access to those files, corresponding to each user, and maintaining the permission? Does Spring's ACL help? Or do we have to create permission group for each user? I am totally cool with the filesystem url. My only concern is with starting a seperate process (chmod and stuff), using something like ProcessBuilder to run Operating Systems commands (or is there better solution ?). And what about permissions?


Q: Serverless Web App Architecture I am currently designing a simple Serverless Web App using Serverless.
My current expected stack is;

*

*API Gateway

*Lambda

*DynamoDB

*Static Single Page App

I have followed a few tutorials for building the Serverless API, Lambda and DynamoDB using the Serverless Framework and I have built my single page app however right now they are 2 separate entities.
What I am looking to do now is bring the static site (nodejs) into the same project as my API, Lambda and DynamoDB and use the Serverless Framework to control the deployment however I'm struggling to find guidance on;

*

*How do you represent the static web site part in the Serverless
Framework

*How best to host that static web site (e.g. s3 static site hosting
or other option?)

*How best to reference a API Gateway URL that is being generated at
the same time the deployment is happening (e.g. via Serverless
Framework)

Could anyone provide any insight into how this is supposed to work, or maybe point me int he direction of some good blogs/resources?
Kind Regards,
John

A: I had a SPA (single page application) written in React communicating to a REST JSON API written in nodejs and hosted on Heroku as a monolith.
I migrated to AWS Lambda and split the monolith into 3+ AWS Lambdas micro services inside of a monorepo
The following project structure is good if your SPA requires users to login to be able to do anything.
I used a single git repository where I have a folder for each service:

*

*api

*app

*www

Inside of each of the services' folders I have a serverless.yml defining the deployment to a separate AWS Lambda.
Each service maps to only 1 function index which accepts all HTTP endpoints. I do use 2 environments staging and production.
My AWS Lambdas are named like this:

*

*example-api-staging-index

*example-api-production-index

*example-app-staging-index

*example-app-production-index

*example-www-staging-index

*example-www-production-index

I use AWS Api Gateway's Custom Domain Names to map each lambda to a public domain:

*

*api.example.com (REST JSON API)

*app.example.com (single page application that requires login)

*www.example.com (server side rendered landing pages)

*api-staging.example.com

*app-staging.example.com

*www-staging.example.com

You can define the domain mapping using serverless.yml Resources or a plugin but you have to do this only once so I did manually from the AWS website console.
My .com domain was hosted on GoDaddy but I migrated it to AWS Route 53 since HTTPS certificates are free.
app service

*

*/bundles-production

*/bundles-staging

*src (React/Angular single page application here)

*handler.js

*package.json

*serverless.yml

The app service contains a folder /src with the single page application code. The SPA is built locally on my computer in either ./bundles-production or ./bundles-staging based on the environment. The built generates the .js and .css bundles and also the index.html. The content of the folder is deployed to a S3 bucket using the serverless-s3-deploy plugin when I run serverless deploy -v -s production.
I defined only 1 function getting called for all endpoints in the serverless.yml (I use JSON instead of YAML):
...
    "functions": {

        "index": {
            "handler": "handler.index",
            "events": [
                { "http": "GET /" },
                { "http": "GET /{proxy+}"}
            ]
        },

    },

The handler.js file returns the index.html defined in /bundles-staging or /bundles-production
I use webpack to build the SPA since it integrates very well with serverless with the serverless-webpack plugin.
api service
I used aws-serverless-express to define all the REST JSON API endpoints. aws-serverless-express is like normal express but you can't do some things like express.static() and fs.sendFile().
I tried initially to use a separate AWS Lambda function instead of aws-serverless-express for each endpoint but I quickly hit the CloudFormation mappings limit.
www service
If most of the functionality of your single page application require a login it's better to place the SPA on a separate domain and use the www domain for server-side rendered landing pages optimized for SEO.
BONUS:
graphql service
Using a micro service architecture makes it easy to experiment. I'm currently rewriting the REST JSON API in graphql using apollo-server-lambda

A: I have done pretty much the same architecture and hosted the single page app on s3. What you can do is set up cloudfront for api gateway and than point api.yourDomain.com to that cloudfront. Than you will also need to enable cors on your api.
This plugin handles setting up domain and cloudfront for you:
https://github.com/amplify-education/serverless-domain-manager
I am not sure about your project requirements but if you want to serve static files faster setting up domain->cloudfront->s3 might be a wise choice.

A: Here is something meaningful and worthy to read about the Serverless Architecture and Azure Serverless services
https://sps-cloud-architect.blogspot.com/2019/12/what-is-serverless-architecture.html


Q: cannot import name 'PackageFinder' from 'pip._internal.index' pip3 On my mac, pip3 exists in /usr/bin. When I try to install any package using pip3, it is failing. Even simple 'pip3' also failing with below error.
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/usr/bin/pip3", line 6, in <module>
    from pip._internal import main
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/__init__.py", line 40, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 8, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 11, in <module>
    from pip._internal.commands import (
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/commands/__init__.py", line 6, in <module>
    from pip._internal.commands.completion import CompletionCommand
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/commands/completion.py", line 6, in <module>
    from pip._internal.cli.base_command import Command
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 26, in <module>
    from pip._internal.index import PackageFinder
ImportError: cannot import name 'PackageFinder' from 'pip._internal.index' (/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_internal/index/__init__.py)

I tried to remove /usr/bin/pip3. But it is saying operation is not permitted. I am unable to remove pip3. Can any one please let me know how to fix this issue.
Note: I went through some stack overflow questions related to this. Most of the answers suggested to uninstall and install pip3. But I am unable to remove pip3.


Q: jquery: Callback not received on clicking of button I need to send a request on click of button but callback is not received on firing of click event of the button.
Following is code snippet:
$(document).ready(function () {
    var counter = 0;
    $("#trail").click(function () {
        $("#dialog").dialog();
        if (counter < 1) {
            $("#searchboxdiv").after('<input type="text" id="searchbox">');
            $("#searchbox").after('<input type="button" id="searchbutton" value="search">');
            counter++;
        }
    });
    $("#searchbutton").click(function () {
        var dataToSend = null;
        $.ajax({
            data: dataToSend,
            url: "FormHandler",
            success: function (result) {},
            beforeSend: function () {
                dataToSend = $("#searchbox").val();
            }
        });
    });
    $("#searchboxdiv").on('click', "#searchbutton", function(){
        var data = null;

    });
});

I added the textbox in the dialog box dynamically and on click of button in dialog box, callback is not received

A: Use event delegation (for dynamically added #searchbutton)
 $('#searchboxdiv').on('click',"#searchbutton",function(){

*

*http://learn.jquery.com/events/event-delegation/

A: Your options:

*

*Use event delegation. Bind the click to immediate static parent like this :
 $("#searchboxdiv").on('click', "#searchbutton", function(){ });

*Or, bind it to the document.
 $(document).on('click', "#searchbutton", function(){ });

*Or, move the existing click after counter++;, ie., inside  $("#trail")'s click handler.
For more info, see on()


Q: Is there a method to setup flink memory dynamically? When using Apache Flink we can configure values in flink-conf.yaml. But here using CLI commands we can assign some values Dynamically when starting or submitting a job or a task in flink.
eg:- bin/taskmanager.sh start-foreground -Dtaskmanager.numberOfSlots=12

But some values like jobmanager.memory.process.size and taskmanager.memory.process.size are unable to set Dynamically using "-D".
Is there a way to set up those values dynamically when starting jobmanager and taskmanager using CLI?

A: May be this might help you :
ParameterTool parameters = ParameterTool.fromPropertiesFile("src/main/resources/application.properties"); // one can specify the properties defined in conf/flink-conf.yaml in this properties file
Configuration config = Configuration.fromMap(parameters.toMap());
TaskExecutorResourceUtils.adjustForLocalExecution(config);

StreamExecutionEnvironment env =
        StreamExecutionEnvironment.getExecutionEnvironment(config);
env.setParallelism(3);

System.out.println("Config Params : " + config.toMap());

Make a note to set the parallelism equal to the number of task slots for task manager as per this link. By default, the number of task slots for a task manager is one.


Q: Fetching npm module from GitHub brings ".git" folder I recently forked an npm package and updated it for my needs. Then, I changed the dependency on packages.json to point to my GitHub repo, and it worked fine. But, when npm installed the module, it brought also the git folder (.git). Because of that, when I try to install anything else, npm gives me this error:
npm ERR! path /node_modules/react-native-static-server
npm ERR! code EISGIT
npm ERR! git /node_modules/react-native-static-server: Appears to be a git repo or submodule.
npm ERR! git /node_modules/react-native-static-server
npm ERR! git Refusing to remove it. Update manually,
npm ERR! git or move it out of the way first.

What am I doing wrong? How do I avoid the .git folder from being downloaded?
You can check the repo here: https://github.com/dccarmo/react-native-static-server
EDIT
The dependency in my packages.json:
"react-native-static-server": "dccarmo/react-native-static-server"

A: This seems to be an old question, but I was running into the same thing today.
I am rather new to git and npm, but I did find something that may be of help to someone.
If the git repo does not have a .gitignore, the .git folder is not downloaded / created.
If the git repo does have a .gitignore, the .git folder is downloaded / created.
I had two repos, one without a .gitignore (because when I made it I was not aware of what .gitignore was or did), and one with a .gitignore. I included both as npm packages in a project, and noticed that the one without the .gitignore was not giving me the EISGIT error (because of the .git folder).
So, after I found this question, I removed the .gitignore from that repo and now it too does not create a .git folder.
Later on, I discovered that adding both a .gitignore and a .npmignore to the project now stops the .git folder from appearing. I did not add .git in my .npmignore.


Q: Linux: access i2c device within board_init function (iMX6 SOC running Linux 3.0)
I need to run a few I2C transactions in my board_init function. I tried calling i2c_get_adapter, then i2c_transfer, those being available in kernel mode, but i2c_get_adapter returns NULL. It's already called imx6q_add_imx_i2c, which is a wrapper around platform_device_register_full, but that isn't enough.
I can manipulate GPIO in board_init by calling gpio_request to obtain access, and gpio_free at the end. Is there something analogous to those functions for i2c?
--- added details ---
I've got a LAN9500A USB 100Base-T Ethernet MAC connected to a LAN9303 3-port switch with a virtual PHY. The MAC has a GPIO reset line that has to be turned off before it will spring to life and enumerate on the USB. That's done in board_init because it's completely nonstandard, so we don't want to patch the stock driver to manipulate some GPIO that's not part of the device.
The problem I'm having is that even though the MAC is permanently attached to the VPHY, it's not noticing it's connected, and an "ip link show eth1" command shows NO-CARRIER. I found I can kickstart it by unmasking the VPHY's Device Ready interrupt via I2C, but I also have to mask it immediately, or I get infinite interrupts. That's not the right solution, but Microchip has been no help in showing me a better way. But we don't want to patch the MAC driver with code to fiddle with the PHY.
There is no PHY driver for this part, and the MII interface to the VPHY doesn't include any interrupt-related registers, so it must be done through I2C. Writing a PHY driver just to flip a bit on and off once seems a lot of trouble, especially for a newbie like me who's never written a driver before.
I can do it in Python in a startup script, but that, too, seems like a heavyweight solution to a lightweight problem.

A: That's a wrong approach. Board file supposed to register device drivers and pass important information to them, rather than act as a device driver itself. I'm not sure if what you're trying to do is even possible.
If you really need to extract something from your I2C device on a very early stage - do that in the bootloader and pass the data to kernel via cmdline (U-boot, by the way, has I2C support for a quite some time). Then later, kernel might do appropriate actions depending on what you have passed to it.


Q: Calling methods from objects which implement an interface I am trying to wrap my head around interfaces, and I was hoping they were the answer to my question.
I have made plugins and mods for different games, and sometimes classes have onUpdate or onTick or other methods that are overridable.
If I make an interface with a method, and I make other classes which implement the method, and I make instances of the classes, then how can I call that method from all the objects at once?

A: You'll be looking at the Observer pattern or something similar. The gist of it is this: somewhere you have to keep a list (ArrayList suffices) of type "your interface". Each time a new object is created, add it to this list. Afterwards you can perform a loop on the list and call the method on every object in it.
I'll edit in a moment with a code example.
public interface IMyInterface {
    void DoSomething();
}

public class MyClass : IMyInterface {
    public void DoSomething() {
         Console.WriteLine("I'm inside MyClass");
    }
}

public class AnotherClass : IMyInterface {
    public void DoSomething() {
         Console.WriteLine("I'm inside AnotherClass");
    }
}

public class StartUp {
    private ICollection<IMyInterface> _interfaces = new Collection<IMyInterface>();

    private static void Main(string[] args) {
        new StartUp();
    }

    public StartUp() {
        AddToWatchlist(new AnotherClass());
        AddToWatchlist(new MyClass());
        AddToWatchlist(new MyClass());
        AddToWatchlist(new AnotherClass());
        Notify();
        Console.ReadKey();
    }

    private void AddToWatchlist(IMyInterface obj) {
        _interfaces.Add(obj);
    }

    private void Notify() {
        foreach (var myInterface in _interfaces) {
            myInterface.DoSomething();
        }
    }
}

Output:
I'm inside AnotherClass
I'm inside MyClass
I'm inside MyClass
I'm inside AnotherClass

Edit: I just realized you tagged it as Java. This is written in C#, but there is no real difference other than the use of ArrayList instead of Collection.

A: An interface defines a service contract. In simple terms, it defines what can you do with a class.
For example, let's use a simple interface called ICount. It defines a count method, so every class implementing it will have to provide an implementation.
public interface ICount {
    public int count();
}

Any class implementing ICount, should override the method and give it a behaviour:
public class Counter1 implements ICount {
    //Fields, Getters, Setters
    @Overide
    public int count() {
        //I don't wanna count, so I return 4.
        return 4;
    }
}

On the other hand, Counter2 has a different oppinion of what should count do:
public class Counter2 implements ICount {
    int counter; //Default initialization to 0
    //Fields, Getters, Setters
    @Overide
    public int count() {
        return ++count;
    }
}

Now, you have two classes implementing the same interface, so, how do you treat them equally? Simple, by using the first common class/interface they share: ICount.
ICount count1 = new Counter1();
ICount count2 = new Counter2();
List<ICount> counterList = new ArrayList<ICount>();
counterList.add(count1);
counterList.add(count2);

Or, if you want to save some lines of code:
List<ICount> counterList = new ArrayList<ICount>();
counterList.add(new Counter1());
counterList.add(new Counter2());

Now, counterList contains two objects of different type but with the same interface in common(ICounter) in a list containing objects that implement that interface. You can iterave over them and invoke the method count. Counter1 will return 0 while Counter2 will return a result based on how many times did you invoke count:
for(ICount current : counterList)
    System.out.println(current.count());

A: You can't call a method from all the objects that happen to implement a certain interface at once. You wouldn't want that anyways. You can, however, use polymorphism to refer to all these objects by the interface name. For example, with
interface A { }
class B implements A { }
class C implements A { }

You can write
A b = new B();
A c = new C();

A: Interfaces don't work that way. They act like some kind of mask that several classes can use. For instance:
public interface Data {
    public void doSomething();
}

public class SomeDataStructure implements Data {
    public void doSomething()
    {
        // do something
    }
}

public static void main(String[] args) {
    Data mydataobject = new SomeDataStructure();
}

This uses the Data 'mask' that several classes can use and have certain functionality, but you can use different classes to actually implement that very functionality.

A: The crux would be to have a list that stores every time a class that implements the interface is instantiated. This list would have to be available at a level different that the interface and the class that implements it. In other words, the class that orchestrates or controls would have the list.
An interface is a contract that leaves the implementation to the classes that implements the interface. Classes implement the interface abide by that contract and implement the methods and not override them.
Taking the interface to be
 public interface Model {
        public void onUpdate();
        public void onClick();
       }

    public class plugin implements Model {

        @Override
        public void onUpdate() {
            System.out.println("Pluging updating");

        }

        @Override
        public void onClick() {
            System.out.println("Pluging doing click action");

        }
    }

Your controller class would be the one to instantiate and control the action
 public class Controller {

    public static void orchestrate(){
        List<Model> modelList = new ArrayList<Model>();
        Model pluginOne  = new plugin();
        Model plugTwo  = new plugin();
        modelList.add(pluginOne);
        modelList.add(plugTwo);

        for(Model model:modelList){
            model.onUpdate();
            model.onClick();
        }
    }
}

You can have another implementation called pluginTwo, instantiate it, add it to the list and call the methods specified by the interface on it.


Q: Why does multiprocessing use only a single core after I import numpy? I am not sure whether this counts more as an OS issue, but I thought I would ask here in case anyone has some insight from the Python end of things.
I've been trying to parallelise a CPU-heavy for loop using joblib, but I find that instead of each worker process being assigned to a different core, I end up with all of them being assigned to the same core and no performance gain.
Here's a very trivial example...
from joblib import Parallel,delayed
import numpy as np

def testfunc(data):
    # some very boneheaded CPU work
    for nn in xrange(1000):
        for ii in data[0,:]:
            for jj in data[1,:]:
                ii*jj

def run(niter=10):
    data = (np.random.randn(2,100) for ii in xrange(niter))
    pool = Parallel(n_jobs=-1,verbose=1,pre_dispatch='all')
    results = pool(delayed(testfunc)(dd) for dd in data)

if __name__ == '__main__':
    run()

...and here's what I see in htop while this script is running:

I'm running Ubuntu 12.10 (3.5.0-26) on a laptop with 4 cores. Clearly joblib.Parallel is spawning separate processes for the different workers, but is there any way that I can make these processes execute on different cores?

A: Python 3 now exposes the methods to directly set the affinity
>>> import os
>>> os.sched_getaffinity(0)
{0, 1, 2, 3}
>>> os.sched_setaffinity(0, {1, 3})
>>> os.sched_getaffinity(0)
{1, 3}
>>> x = {i for i in range(10)}
>>> x
{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
>>> os.sched_setaffinity(0, x)
>>> os.sched_getaffinity(0)
{0, 1, 2, 3}

A: After some more googling I found the answer here.
It turns out that certain Python modules (numpy, scipy, tables, pandas, skimage...) mess with core affinity on import. As far as I can tell, this problem seems to be specifically caused by them linking against multithreaded OpenBLAS libraries.
A workaround is to reset the task affinity using
os.system("taskset -p 0xff %d" % os.getpid())

With this line pasted in after the module imports, my example now runs on all cores:

My experience so far has been that this doesn't seem to have any negative effect on numpy's performance, although this is probably machine- and task-specific .
Update:
There are also two ways to disable the CPU affinity-resetting behaviour of OpenBLAS itself. At run-time you can use the environment variable OPENBLAS_MAIN_FREE (or GOTOBLAS_MAIN_FREE), for example
OPENBLAS_MAIN_FREE=1 python myscript.py

Or alternatively, if you're compiling OpenBLAS from source you can permanently disable it at build-time by editing the Makefile.rule to contain the line
NO_AFFINITY=1

A: This appears to be a common problem with Python on Ubuntu, and is not specific to joblib:

*

*Both multiprocessing.map and joblib use only 1 cpu after upgrade from Ubuntu 10.10 to 12.04

*Python multiprocessing utilizes only one core

*multiprocessing.Pool processes locked to a single core
I would suggest experimenting with CPU affinity (taskset).


Q: Can't uninstall npm module on OSX I have installed cordova on my mac and want to uninstall it.
I've tried the following:
npm uninstall cordova -g

but I get the following error:
npm WARN uninstall not installed in /usr/local/Cellar/node/0.10.32/lib/node_modules: "cordova"

Any ideas?

A: There are know issues with the way homebrew and npm play together. From this article by Dan Herbert

There's an NPM bug for this exact problem. The bug has been "fixed" by
  Homebrew installing npm in a way that allows it to manage itself once
  the install is complete. However, this is error-prone and still seems
  to cause problems for some people. The root of the the issue is really
  that npm is its own package manager and it is therefore better to have
  npm manage itself and its packages completely on its own instead of
  letting Homebrew do it.

Aside from that your node version is out of date. If you can upgrade you should do so to Node v4.1.2. Follow the instructions on the node.js site.


Q: Sum of arrays between specific positions in the array I have two arrays of doubles. For example: {1,2,3} {4,5,6} We are using (x,y) coordinates as (1,4),(2,5) and (3,6) for example. So there are 3 data points in this example. We need to calculate the slope between all of the points and store the slope with the maximum absolute value. To do this I have a function with a 'for' loop inside a 'for' loop. My code is successfully calculating the maximum absolute value slope when I use it in the Main.
We need to find the positions that this max slope occurs at and print out the corresponding positions in array2. I have done this.
Finally there is a third array. We need to calculate the average value of the numbers up to the first point used in the max slope, the average value of the numbers between the first and second points used in the max slope and also the average value of the numbers from the second point used in the max slope to the end of the array.
Does this needs to be done in the main, or does it need its own method?
public static void Main()
{
    List<double> Array1 = new List<double>();
    List<double> Array2 = new List<double>();
    List<double> Array3 = new List<double>();

    Array1.Add(Convert.ToDouble(columns[0]));
    Array2.Add(Convert.ToDouble(columns[1]));
    Array3.Add(Convert.ToDouble(columns[1]));

    int[] positions = GreatestSlopeLocation(Array1, Array2);

    Console.WriteLine("The Location of the max slope is {0} {1}",
        Array2[positions[0]], Array2[positions[1]]);
}

static int[] GreatestSlopeLocation(List<double> x, List<double> y)
{
    int size = x.Count;
    double maxSlope = Double.MinValue; // don't set this to 0
    // consider case when all slopes are  negative
    //
    int index1 = 0;
    int index2 = 0;

    for (int i = 0; i < size; i++)
    {
        for (int j = i + 1; j < size; j++)
        {
            double slope = Math.Abs((y[j] - y[i]) / (x[j] - x[i]));
            if (slope > maxSlope) // I think you want > instead of < here
            {
                maxSlope = slope;
                index1 = i;
                index2 = j;
            }
        }
    }
    return new int[]{index1, index2};
}


Q: Firebase hosting using Old webpack bundle when deployed via GitHub Actions I'm trying to create a GitHub-Action that first uses Webpack to create a bundle.js file and then deploy my index.html and newly created bundle.js to Firebase hosting.
First I just pushed my bundle.js in my GitHub repository, but I since removed it via "git rm --cache public/bundle.js" (not sure if this is the exact command) and added bundle.js to my .gitignore.
The action does work (no error or crash) but it seems that Firebase does not use the newly packaged bundle.js, but instead uses an old version that might be cached somewhere?
firebase-hosting-pull-request.yml:
name: Deploy to Firebase Hosting on PR
'on': pull_request
jobs:
  build_and_preview:
    if: '${{ github.event.pull_request.head.repo.full_name == github.repository }}'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - run: npm ci && npm run build
      - uses: FirebaseExtended/action-hosting-deploy@v0
        with:
          repoToken: '${{ secrets.GITHUB_TOKEN }}'
          firebaseServiceAccount: '${{ secrets.FIREBASE_SERVICE_ACCOUNT_PROJECTLIBRARIAN }}'
          projectId: projectlibrarian

npm run build in this case executes webpack --mode=production
My index.js (which is webpack'd into bundle.js) only contains a single log statement which I changed to test the GitHub-Action.
Link to the Repo if it helps someone
I'm wondering if I need to clear some kind of GitHub actions cache? or maybe the Firebase deploy action script I'm using isn't doing what I think it should be doing? What confusing me the most is where does Firebase get the old version of bundle.js!

A: Ok I have found two changes that fixed the problem.
Add channelId: live in my workflow yaml
Reload the site with the "Empty chache and Hard reload" developer option.


Q: Javascript: List Files in Directory on Server Is there a way to list all files in a particular directory on the server using Javascript? If so, how? I  am not allowed to use platforms which require server side installation. I mean platforms like node.js and such.

A: Well how would you list the server side files if you have no access to server side ?
"I am not allowed to use platforms which require server side installation."
There are some workarounds for this problem, but these usually needs some extra enabled flags from the user side. http://www.chrome-allow-file-access-from-file.com/

A:
I am not allowed to use platforms which require server side installation.

You need some server side handler for this request. But may be you already have something. For example mod_autoindex for apache..


Q: How do I remove duplicate values in a v-for array? I coded like this to remove duplicate values.

vue

       {{member.mem_name}}

vue (script)

  computed: {
    uniqueMemName() {
      return _.uniqBy(this.members, function(m) {
        return m.mem_name;
    });
  }
},

I also installed lodash. But I get an error. Which part is wrong?
Please let me know if there is another method other than the one I wrote.
++) error console
console window
++) array information:
I have tables A and B. Table A imports only the mem_name column. Table B imports all columns.
Example ->

a.mem_name
b.col1
b.col2

mem1
10
20

mem1
30
40

mem2
50
60

I'm working on making duplicate mem_names into one at this time. Using lodash's unique features.

A: If you want to use lodash just for this, which sounds like the case, I suggest that there may be a better way without it, only using newer vanilla JS:
...
  computed: {
    uniqueMemName() {
      return [...new Set(this.members.map(m => m.mem_name))]
    }
  }

Sets are always unique, so mapping and converting to a set and then back to array gives us a unique array.